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Executive  Summary 


Who  Should  Read  This  Report  and  Why?  DoD  personnel  who  develop  Information 
Teehnology  systems  and  users  of  the  Military  Airspaee  Management  System  (the 
System)  should  be  interested  in  the  progress  of  the  program.  The  report  diseusses 
Defense  Hotline  allegations  regarding  the  System  as  well  as  the  use  and  aeeeptanee  of 
the  System  within  DoD. 

Introduction.  This  audit  resulted  from  Defense  Hotline  allegations  on  the  System. 
Speeifieally,  the  Defense  Hotline  eomplainant  submitted  seven  allegations  relating  to 
the  effieieney  and  effeetiveness  of  the  System.  Appendix  B  addresses  the  allegations 
and  the  results  of  our  review  in  more  detail.  DoD  developed  the  Internet-based  system 
to  provide  the  eapability  to  effieiently  sehedule,  manage,  doeument,  and  report  military 
airspaee  use  data.  The  System  is  to  interfaee  with  seleeted  existing  Speeial  Use 
Airspaee  seheduling  systems  and,  where  feasible,  provide  a  eonsolidated  database  of 
sehedule  data  and  seheduling  eapability  for  nonautomated  users.  In  addition,  the 
System  will  interfaee  with  the  Federal  Aviation  Administration  for  airspaee  use 
reporting. 

Results.  After  10  years  of  development  and  fielding,  and  expenditures  of  $24  million, 
the  System  has  not  met  its  original  objeetive  of  being  a  DoD-wide  system  for  managing 
military  airspaee  seheduling  funetions  and  providing  usage  information  to  DoD  and  the 
Federal  Aviation  Administration.  The  Serviees  and  seheduling  ageneies  were  not  using 
the  System,  the  System  was  being  asked  to  do  too  many  funetions  for  too  many 
different  types  of  airspaee,  it  was  too  eumbersome  for  easy  use,  and  System  airspaee 
usage  reports  did  not  fulfill  the  reporting  requirements  of  other  ageneies.  Training  was 
also  inadequate  beeause  of  limited  funding  and  resourees,  a  laek  of  eoordination  from 
the  user  eommunity,  and  diseord  between  the  evolving  System  and  eurrent  training 
materials.  In  addition,  the  funds  that  the  Air  Foree  spent  on  the  Military  Airspaee 


Management  System  were  not  effeetive  beeause  the  System  had  not  met  many  Serviee 
requirements.  Survey  responses  from  193  of  621  users  of  the  system  showed  that:^ 

•  34  pereent  (66  of  the  193)  said  it  had  the  required  funetionality, 

•  37  pereent  (71  of  the  193)  said  it  was  user  friendly,  and 

•  38  pereent  (73  of  the  193)  would  reeommend  the  System. 

Management  Comments  and  Audit  Response.  The  Assistant  Seeretary  of  Defense 
(Command,  Control,  Communieations,  and  Intelligenee)  eoneurred  with  the 
reeommendations,  but  did  not  provide  an  aetion  plan  and  eompletion  dates  for 
implementing  two  of  the  reeommendations.  The  Air  Foree  provided  unsolieited 
comments.  The  Air  Force  agreed  with  the  recommendations,  but  commented  that  it 
would  be  hesitant  to  agree  with  allowing  the  other  Services  to  interface  directly  with  the 
Federal  Aviation  Administration,  due  to  safety  concerns.  Additionally,  the  Air  Force 
commented  that  minor  upgrades  and  technical  refreshment  efforts  would  be  required  to 
ensure  that  the  System  remains  operational.  We  agree  that  safety  concerns  should  be 
considered  when  deciding  whether  the  other  Services  should  interface  directly  with  the 
Federal  Aviation  Administration  in  scheduling  airspace.  Also,  we  recognize  that  the 
Air  Force  may  need  to  make  minor  expenditures  to  keep  the  Military  Airspace 
Management  System  operational  until  the  Assistant  Secretary  of  Defense  (Command, 
Control,  Communications,  and  Intelligence)  and  the  Air  Force  determine  the  most  cost- 
effective  alternative  for  scheduling  military  airspace.  We  request  that  the  Assistant 
Secretary  of  Defense  (Command,  Control,  Communications,  and  Intelligence)  provide, 
by  July  25,  2002,  an  action  plan  with  dates  of  completion  for  establishing  an  in-process 
action  team  to  determine  whether  the  System  is  the  most  cost-effective  scheduling  tool 
to  meet  future  DoD  scheduling  needs  and  for  tasking  the  Services  to  standardize 
scheduling  procedures.  A  discussion  of  the  management  comments  is  in  the  Finding 
section  of  the  report  and  the  complete  text  is  in  the  Management  Comments  section. 


‘  Note:  Percentages  reflect  actual  responses  to  each  question  and  may  not  represent  the  193  surveys 
returned  or  the  621  account  holders  surveyed. 
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Background 


Allegations.  The  Inspector  General  of  the  Department  of  Defense  received 
seven  Defense  Hotline  allegations  concerning  the  Military  Departments’ 
development  and  use  of  the  Military  Airspace  Management  System  (MAMS). 
Appendix  B  addresses  those  allegations  and  the  results  of  our  review. 

System  Description.  The  MAMS  is  an  Internet-  and  software-based  system 
intended  to  provide  the  Services  with  a  capability  to  efficiently  schedule  Special 
Use  Airspace  (SUA)  DoD-wide  and  to  accurately  document  and  report  Service 
use  of  SUA  to  the  Federal  Aviation  Administration  (FAA).  MAMS  includes  a 
centralized  data  automation  bank  based  on  a  distributed  wide  area  computer 
network  that  is  intended  to  interconnect  with  other  DoD  scheduling  systems  and 
appropriate  FAA  facilities.  The  system  is  to  serve  as  a  management  and 
scheduling  system  to  effectively  use  military  SUA,  to  document  and  report  the 
use  of  that  airspace  to  FAA,  and  to  release  unused  SUA  for  joint  use. 

At  the  time  of  the  Joint  Operational  Requirements  Document,  September  20, 
1993,  DoD  was  unable  to  provide  near  real-time  scheduling  and  management  of 
airspace  or  near  real-time  airspace  usage  data  collection  and  reporting  because 
of  data  processing  constraints.  DoD  intended  MAMS  to  work  with  the  FAA 
Special  Use  Airspace  Management  System  (SAMS),  a  subsystem  of  the  Military 
Operations  Systems.  The  mission  of  the  SAMS  is  to  provide  the  most  efficient 
use  of  airspace  while  providing  effective  coordination  between  DoD  and  FAA  to 
ensure  the  maximum  level  of  safety  for  the  users. 

Diversity  of  Special  Use  Airspace.  Within  the  continental  United  States,  there 
are  two  categories  of  Military  Airspace.  The  first  category  is  SUA  that  includes 
warning  areas,  military  operations,  and  restricted  areas.  It  also  includes  areas 
where  flight  is  prohibited  at  any  time,  such  as  the  White  House  and  various 
monuments.  The  second  category  is  Other  Airspace  that  includes  military 
training  routes,  air  traffic  control-assigned  airspace,  aerial  refueling  anchors, 
aerial  refueling  tracks,  and  altitude  reservations. 

MAMS  Development.  The  Joint  Program  Office  developed  MAMS  for  DoD  in 
response  to  two  General  Accounting  Office  (GAO)  reports  that  documented  the 
need  for  comprehensive  long-range  airspace  planning  and  the  need  for  DoD  and 
other  agencies  to  submit  standardized  usage  reports  to  the  FAA  to  support  their 
use  of  SUA.  Specifically: 

•  Report  No.  NSIAD-87-93,  “Better  Planning  Is  Needed  to  Meet 
Future  Requirements,”  March  23,  1987,  documents  that  DoD  needed 
comprehensive  long-range  airspace  planning  to  help  ensure  that  the 
Services  meet  future  airspace  requirements.  The  report  recommends 
that  the  Services  develop  comprehensive  airspace  plans  that  defined, 
validated,  and  supported  their  future  airspace  requirements. 

•  Report  No.  RCED-88-147,  “FAA  Needs  to  Improve  Its  Management 
of  Special  Use  Airspace,”  August  5,  1988,  reviews  FAA 
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management  of  SUA,  whieh  is  primarily  used  for  military  training. 
The  GAO  found  that  the  FAA  was  not  effeetively  managing  SUA  to 
ensure  effieient  and  appropriate  use.  The  report  reeommends  that  the 
Seeretary  of  Transportation  direet  the  FAA  to  require  standardized 
user  reporting  of  aetual  usage  data  for  restrieted  areas,  periodie 
review  of  the  usage  reports,  and  establishment  of  standards  for 
measuring  the  effeetiveness  of  SUA  utilization. 

On  June  11,  1987,  the  Department  of  the  Air  Foree  validated  a  Statement  of 
Operational  Need  for  MAMS.  The  Air  Foree  then  tasked  MITRE  to  develop  an 
airspaee  management  prototype.  In  Oetober  1989,  MITRE  developed  the 
prototype  under  letter  eontraet  F19628-89-C-0001  at  a  eost  of  $750,000.  The 
MITRE  prototype  used  a  Unix  operating  system  speeifieally  for  seheduling  SUA 
at  Edwards  Air  Foree  Base,  California.  On  Mareh  16,  1993,  the  Air  Foree 
sueeessfully  eondueted  a  MAMS  prototype  demonstration  at  Edwards  Air  Foree 
Base.  A  Designated  Aequisition  Commander  memorandum,  April  24,  1994, 
gave  approval  for  MAMS  to  enter  into  Phase  II,  Engineering  and  Manufaeturing 
Development  phase  of  the  aequisition  proeess. 

On  February  24,  1992,  the  Air  Foree  Eleetronie  Systems  Center  advertised  the 
MAMS  notiee  of  eontraet  aetion  (solieitation)  in  the  Business  Commerce  Daily. 
The  Air  Foree  Eleetronie  Systems  Center  modified  the  solieitation  six  times 
before  awarding  the  eontraet  beeause  of  ehanges  to  the  aequisition  strategy, 
eontraet  type,  and  hardware.  The  solieitation  proeess  took  about  27  months. 

On  June  8,  1994,  the  Air  Foree  Eleetronie  Systems  Center  awarded  an 
engineering  and  manufaeturing  development  eontraet  to  Computer  Based 
Systems,  Ineorporated,  in  Fairfax,  Virginia.  Computer  Based  Systems 
Ineorporated  experieneed  systems  development  problems  and  the  Air  Foree 
Eleetronie  Systems  Center  terminated  the  eontraet  for  eonvenienee  on 
Deeember  19,  1996,  at  a  eost  of  $1,220,980.  Prior  to  terminating  the  eontraet 
with  Computer  Based  Systems  Ineorporated,  the  Air  Foree  Eleetronie  Systems 
Center  awarded  a  sole-souree  eontraet  to  Hughes  as  delivery  order  eontraet 
F04606-95-D-0070-0016,  effeetive  November  3,  1995,  in  the  amount  of 
$11.4  million.  In  1997,  Raytheon  aequired  Hughes  and  eontinued  to  use  the 
same  faeility  and  employees  in  Lanham,  Maryland,  for  the  MAMS  projeet. 
However,  Raytheon  proposed  an  alternative  approaeh  to  develop  MAMS  using 
an  Internet-based,  windows  platform  program,  whieh  the  program  offiee 
aeeepted.  The  MAMS  beeame  operational  at  Tinker  Air  Foree  Base, 

Okl^oma,  in  Oetober  2000. 

MAMS  Sustainment.  Air  Foree  performs  the  MAMS  sustainment  effort  for 
DoD.  The  sustainment  projeet  management  offiee  and  the  MAMS  Central 
Faeility  are  loeated  and  managed  at  Tinker  Air  Foree  Base,  Oklahoma.  MAMS 
was  an  Aequisition  Category  III  program.  On  September  14,  2000,  Air  Foree 
Eleetronie  Systems  Center  awarded  Computer  Hi-teeh  Management,  Ine.,  and 
Raytheon  Information  Teehnology  and  Seientifie  Serviees  sustainment  eontraets 
through  the  General  Serviees  Administration.  Computer  Hi-teeh  Management, 
Ine.,  provides  database  administration,  helpdesk  support  serviees,  and  training. 
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while  the  Raytheon  Information  Teehnology  and  the  Seientifie  Serviees  provides 
development,  implementation,  and  maintenanee  support  for  the  MAMS  system. 


Survey  Questionnaires.  To  help  aeeomplish  our  audit  objeetive,  we  developed 
and  distributed  survey  questionnaires  to  the  Serviee  MAMS  users.  The  MAMS 
survey  was  designed  primarily  to  obtain  “yes”  or  “no”  responses  and  to  provide 
users  with  the  opportunity  to  submit  written  feedbaek  regarding  MAMS.  The 
results  of  the  survey  also  helped  determine  the  validity  of  the  Defense  Hotline 
allegations. 

Although  the  MAMS  Coneept  of  Operations  states  that  approximately  250  DoD 
units  have  authority  to  sehedule  SUA,  the  program  offiee  states  the  aetual 
number  of  units  is  255.  We  reeeived  a  list  of  1,023  aeeount  holders  from  the 
projeet  management  offiee,  whieh  ineluded  621  aetive  individual  aeeount 
holders  and  402  inaetive  individual  aeeount  holders  (prior  aeeounts).  Of  the 
621  aetive  aeeount  holders,  193  Serviee  aeeount  holders  returned  their  survey 
questionnaire.  The  responding  Serviee  aeeount  holders  are  the  airspaee 
requesters,  airspaee  sehedulers,  and  airspaee  managers.  Airspaee  requesters  are 
users  of  airspaee  who  need  SUA  in  order  to  eonduet  their  missions,  airspaee 
sehedulers  who  manage  airspaee  under  their  eognizanee  and  report  its  usage, 
and  airspaee  managers  who  analyze  airspaee  use  and  work  to  aehieve  overall 
system  effieieney. 

Of  the  193  surveys  returned,  we  did  not  reeeive  responses  to  eaeh  of  the 
11  questions  on  all  of  the  survey  questionnaires,  as  they  were  not  always 
required  and,  in  some  eases,  were  not  provided.  The  Air  Foree  aeeounted  for 
165  of  the  193  total  surveys  and  is  the  Serviee  that  predominantly  uses  MAMS. 
For  the  most  part,  the  Navy  and  the  Army  were  not  using  MAMS  beeause  of 
their  unique  requirements.  The  remaining  user  surveys  inelude  13  for  the  Navy, 
8  for  the  Army,  and  7  for  the  Marine  Corps  (see  Appendixes  C  and  D, 
respeetively,  for  the  speeifie  survey  questions  and  for  a  further  breakdown  of 
user  response  by  Serviee  and  user  funetion). 

Objectives 


The  overall  audit  objeetive  was  to  review  and  assess  allegations  eoneerning  the 
effieieney  and  effeetiveness  of  the  Military  Airspaee  Management  System. 
Speeifieally,  we  reviewed  the  training  requirements  for  the  system,  as  well  as 
eapabilities,  limitations  and  use  of  the  system.  See  Appendix  A  for  a  diseussion 
of  the  audit  seope  and  methodology. 
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Military  Airspace  Management  System 
Usage 

The  Military  Airspace  Management  System  (MAMS),  developed  by  the 
Joint  Program  Office  to  satisfy  DoD-wide  requirements  to  manage, 
schedule,  and  document  the  use  of  SUA,  had  not  gained  widespread  use 
and  acceptance  throughout  DoD.  The  following  factors  contributed  to 
the  System’s  lack  of  use  and  acceptance: 

•  MAMS  did  not  meet  many  of  the  Service-unique  schedule 
requirements  for  SUA; 

•  the  Services  lacked  policies  and  procedures  to  standardize  their 
SUA  schedule  process; 

•  each  Service  believed  its  own  airspace  scheduling  system  was 
efficient  and  effective  and  questioned  the  usefulness  of  MAMS; 

•  the  requirement  for  other  DoD  scheduling  systems  to  interface  with 
MAMS  had  not  been  met,  and  the  majority  of  the  FAA  centers 
were  not  using  the  FAA  SAMS,  which  was  designed  to  interface 
with  MAMS; 

•  the  MAMS  program  lacked  a  DoD  functional  sponsor  and  adequate 
funds  for  improvements;  and 

•  MAMS  did  not  meet  all  testing  requirements. 

As  a  result,  after  10  years  of  development  and  fielding,  and  expenditures 
of  more  than  $24  million,  the  MAMS  had  not  met  its  original  objective 
to  become  the  DoD-wide  system  for  managing  military  airspace 
scheduling  functions  and  for  providing  usage  information  to  DoD  and 
FAA. 

Guidance  on  National  Airspace 

The  initial  DoD  Directive  5030. 19,  “DoD  Responsibilities  on  Federal  Aviation 
and  National  Airspace  System  Matters,”  June  22,  1989,  directed  the  Military 
Services  to  develop,  deploy,  and  sustain  MAMS  to  ensure  availability  of 
airspace  and  the  timely  release  of  airspace  when  not  needed.  The  revised  DoD 
Directive  5030.19,  “DoD  Responsibilities  on  Federal  Aviation  and  National 
Airspace  System  Matters,”  June  15,  1997,  designates  the  Assistant  Secretary  of 
Defense  (Command,  Control,  Communications,  and  Intelligence)  to  provide 
policy  and  oversight  of  DoD  interface  with  FAA  on  all  National  Airspace 
System  matters.  The  revised  DoD  Directive  5030.19  also  establishes  the  DoD 
Policy  Board  on  Federal  Aviation  and  designates  Assistant  Secretary  of  Defense 
(Command,  Control,  Communications,  and  Intelligence)  as  the  Chair  of  the 
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Policy  Board.  In  addition,  the  revised  Direetive  requires  operational 
interoperability  of  equipment  between  DoD  and  FAA. 


MAMS  Use  and  Acceptance 


Sinee  the  Air  Foree  fielded  the  first  MAMS  in  Oetober  2000,  it  had  not  gained 
widespread  use  and  aeeeptanee  throughout  DoD.  Based  on  the  survey  results, 
the  Army,  Navy,  and  Marine  Corps  had  limited  or  no  usage  of  the  system.  An 
E-mail  dated  Oetober  19,  2000,  from  the  Navy’s  Head,  Airspaee  and  Air 
Traffie  Control  Programs,  direeted  its  airspaee  requesters,  sehedulers,  and 
managers  not  to  use  MAMS  until  all  their  personnel  were  trained  on  the  system. 
In  addition,  as  of  February  2002,  despite  Air  Foree  direetion  to  use  MAMS, 
only  48  of  1 1 1 ,  or  43  pereent,  of  Air  Foree  survey  respondents  at  SUA  units 
were  using  MAMS.^ 

Factors  Effecting  Use  and  Acceptance  of  MAMS 


The  following  faetors  eontributed  to  the  laek  of  widespread  Serviee  use  and 
aeeeptanee  of  MAMS:  the  System  did  not  meet  many  of  the  Serviee-unique 
requirements  for  SUA;  the  Serviees  laeked  polieies  and  proeedures  to 
standardize  their  SUA  sehedule  proeess  and  believed  that  their  existing  sehedule 
systems  for  SUA  were  effieient  and  effeetive;  Serviee  users  questioned  the 
usefulness  of  MAMS  beeause  the  requirements  for  the  other  DoD  seheduling 
systems  to  interfaee  with  MAMS  had  not  been  met;  the  majority  of  FAA 
Centers  were  not  using  FAA  SAMS,  whieh  was  designed  to  interfaee  with 
MAMS;  the  MAMS  program  laeked  a  DoD  funetional  sponsor  and  adequate 
funds;  and  MAMS  had  not  met  all  testing  requirements. 

Service-Unique  Schedule  Requirements.  The  requirements  for  airspaee 
seheduling  differ  for  eaeh  Serviee,  as  well  as  within  eaeh  Serviee.  Serviees 
sehedule  different  types  of  airspaee  for  different  reasons.  Airspaee  ean  have 
many  unique  types  of  operations  that  are  eondueted  simultaneously.  Examples 
of  Serviee-unique  airspaee  sehedule  requirements  follow. 

Army  and  Marine  Corps  Scheduling.  The  Army  uses  SUA  for  firing 
field  artillery,  mortars,  remotely  piloted  vehiele  operations,  aireraft  ordnanee 
delivery  and  test  flights,  laser  aetivities,  and  various  types  of  researeh  and 
development.  Although  the  Marine  Corps  falls  under  the  Navy,  its  missions  are 
very  similar  to  that  of  the  Army.  The  Army  and  Marine  Corps  sehedule  SUA 
using  the  Range  Faeility  Management  Support  System. 

Navy  Scheduling.  The  Navy  uses  the  Fleet  Area  Control  and 
Surveillanee  Seheduling  System  (FACSKED)  for  seheduling  beeause  MAMS 
does  not  have  the  eapability  to  sehedule  subsurfaee  missions.  The  Navy 


^There  were  165  total  Air  Force  respondents  and  only  111  addressed  this  question  regarding  MAMS  use. 
The  111  may  not  represent  the  193  who  returned  their  surveys  or  the  Air  Force  account  holders  who 
were  surveyed. 
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requires  the  eapability  to  sehedule  surfaee  missions  (starts  on  the  oeean  surfaee 
and  goes  down  to  minus  98  feet)  and  subsurfaee  missions  (starts  at  minus  98  feet 
and  goes  down  to  the  oeean  floor),  where  the  Navy  sehedules  oeean  spaee  like 
airspaee.  Airspaee  starts  at  1  foot  above  the  surfaee  and  goes  up  into  the 
atmosphere. 

Air  Force  Scheduling.  The  Air  Foree  uses  several  t5q)es  of  SUA  in 
support  of  preliminary,  primary,  and  advaneed  flight  training;  training  for 
missions,  sueh  as  eounter  air,  strategie  attaek,  and  surveillanee;  and  researeh, 
development,  and  testing  of  new  equipment.  Some  Air  Foree  units  use  many 
types  of  SUA  and  fly  a  signifieant  number  of  missions  daily.  Beeause  of  the 
different  missions  and  their  reporting  requirements.  Air  Foree  units  eonsider 
MAMS  ineffieient  and  ineffeetive  for  their  use. 

Service  Policies  for  Standardizing  Scheduling  Process.  With  the  exeeption  of 
the  Air  Foree,  the  other  Serviees  laeked  polieies  and  proeedures  for 
standardizing  their  SUA  sehedule  proeess.  Although  the  Air  Foree  was  direeted 
to  use  MAMS,  the  airspaee  seheduling  units  eontinued  to  use  the  existing 
systems.  A  standardized  SUA  seheduling  system  within  eaeh  Serviee  would 
eontribute  signifieantly  to  meeting  the  requirements  of  the  “Joint  Operational 
Requirements  Doeument,”  September  20,  1993,  whieh  requires  other  automated 
seheduling  systems  to  interfaee  with  MAMS  to  report  usage  data.  A  Serviee 
standard  seheduling  system  ean  be  tailored  to  meet  a  unit’s  unique  requirements. 
Standardization  would  enhanee  effieieney  and  effeetiveness  by  requiring  only 
one  interfaee  with  MAMS.  Also,  standardized  Serviee  seheduling  systems 
would  ensure  eontinuity  as  resourees  rotate  from  one  unit  to  another.  At  the 
time  of  the  review,  the  Serviees  used  a  variety  of  meehanisms  to  sehedule  SUA, 
sueh  as  the  legaey  systems  noted  in  Appendix  E,  Aeeess  databases.  Exeel 
spreadsheets,  and  manual  entries  that  made  it  diffieult  to  interfaee  with  MAMS. 

Efficiency  and  Effectiveness  of  Existing  Service  Schednling  Systems  Versns 
MAMS.  To  determine  the  effieieney  and  effeetiveness  of  existing  Serviee 
Airspaee  Seheduling  Systems  and  the  usefulness  of  MAMS,  we  interviewed 
airspaee  sehedulers  and  eolleeted  Serviee  doeumentation  and  guidanee  on  the 
use  of  existing  seheduling  systems  and  MAMS.  Additionally,  we  distributed 
survey  questionnaires  relating  to  user  satisfaetion  to  MAMS  users. 

User  Interviews  and  Service  Docnmentation.  According  to  the 
Services,  existing  scheduling  systems  were  more  efficient  and  effective  than 
MAMS.  Service  users  indicated  that  in  MAMS,  it  takes  12-15  minutes  to  log 
on  the  Internet  site  and  that  after  logging  on,  they  must  complete  13  steps  to 
request  a  mission.  Once  the  request  is  entered,  a  response  on  the  scheduling 
end  must  be  sent  to  accept  or  deny  the  mission.  If  the  scheduling  unit  is  not 
using  MAMS,  or  does  not  use  it  on  a  full-time  basis,  the  request  may  not  be 
approved  in  an  acceptable  amount  of  time.  Conversely,  in  some  current 
scheduling  systems,  it  is  just  a  matter  of  the  requester  calling  the  scheduler  and 
the  scheduler  entering  the  data  into  an  Excel  spreadsheet  to  decide  whether  to 
accept  or  deny  the  mission,  a  matter  of  two  to  three  steps.  Appendix  E  provides 
additional  detail  on  the  various  scheduling  systems  that  the  Services  were  using. 
The  Service  users  also  stated  that  MAMS  does  not  fulfill  their  reporting 
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requirements,  sueh  as  range  use  reporting.  The  range  usage  report  traeks  a 
variety  of  data  assoeiated  with  determining  range  workload  requirements  and 
providing  information  assoeiated  with  environmental  aetions,  sueh  as 
maintaining  reeords  of  all  hazardous  material  (ammunition  and  explosives)  used 
during  training  and  operational  missions.  Range  usage  data  is  needed  for 
residue  elearanee  and  deeontamination  of  ranges.  Additionally,  Air  Foree  users 
stated  that  they  do  not  use  MAMS  beeause  eustomers  are  not  using  the  system  to 
sehedule  missions.  Further,  Air  Foree  units  stated  that  MAMS  is  redundant  and 
eomplex,  and  that  the  inflexibility  of  MAMS  makes  it  manpower  intensive  and 
frustrating  to  use.  In  addition,  the  Air  Foree  users  expressed  eoneerns  about  the 
inability  of  MAMS  to  aeeommodate  short  notiee  ehanges,  down  time,  the 
System’s  slow  eonneetion  and  response  time,  eonfliet  identifieation,  and  FAA 
seheduling  and  usage  reporting. 

Short  Notice  Changes  and  Down  Time.  Aeeount  holders  said 
that  the  System  is  unable  to  easily  aeeommodate  short  notiee  ehanges  and  down 
time.  Both  short  notiee  ehanges  and  down  time  require  manual  notifieation  and 
deeonflietion  (resolving  eonfliets  that  arise  during  mission  seheduling)  for 
mission  aeeomplishment,  whieh  result  in  inereased  workload  (labor  intensive) 
and  deereased  safety  and  effieieney.  Weather  ehanges  and  maintenanee 
requirements  are  examples  of  short  notiee  ehanges  and  down  time. 

Connection  and  Response  Time.  Required  eonneetion  and 
response  time  are  essential  for  MAMS  to  aehieve  its  mission;  however,  the 
users  reported  diffieulty  in  logging  on  to  the  system,  maintaining  a  eonneetion, 
and  working  at  a  reasonable  speed.  In  addition  to  the  survey  responses  to  eaeh 
of  the  11  questions,  we  reeeived  additional  eomments  from  users  and  28  stated 
that  MAMS  is  too  slow.  Major  comments  included  that  the  system  is  labor 
intensive,  very  slow  even  on  powerful  computers,  and  that  the  system  is 
cumbersome  and  too  complicated  for  everyday  use  without  extensive  training. 
However,  the  MAMS  Response-Time  Study,  which  MITRE  performed  in  April 
1999,  indicated  that  the  MAMS  server,  the  MAMS  link  to  the  Internet,  and  the 
MAMS  firewall  are  not  the  cause  of  any  meaningful  response  time  delays.  The 
study  concluded  that  the  user’s  site  must  be  the  source  of  the  performance 
problem  for  those  users  who  experience  consistently  long  response  time  delays. 
The  cause  of  actual  or  perceived  long  response  time  may  be  Aat  the  users’ 
workstations  are  incorrectly  configured. 

Conflict  Identification.  The  MAMS  conflict  identification  tool 
operates  properly  and  identifies  various  categories  of  conflicts;  however,  the 
system  allows  the  scheduler  to  “override”  the  safety  of  flight  conflict  and 
approve  the  mission.  Some  of  the  conflict  categories  are  safety  of  flight  points 
where  instrument  routes  cross  each  other;  for  example,  one  aircraft  overtaking 
another  on  the  same  route.  Another  example  is  the  time  that  schedulers  take  to 
resolve  conflicts  for  airspace  that  is  subdivided.  Subdivisions  are  designed  to 
activate  together  to  create  different  altitudes  (higher  to  lower),  which  are  used  in 
training  exercises.  When  a  unit  schedules  a  subdivision  for  training,  the  unit 
activates  one  airspace,  but  the  MAMS  recognizes  the  subdivision  as  five 
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different  independent  airspaees.  A  subdivision  sehedule  eonfliet  shows  up  five 
times  in  MAMS,  requiring  the  seheduler  to  resolve  eaeh  eonfliet  separately 
ereating  additional  time  and  work  for  the  seheduler. 

FAA  Scheduling  and  Usage  Reporting.  MAMS  should 
generate  airspaee  sehedules  and  transmit  required  usage  reports  to  FAA; 
however,  the  users  were  not  seheduling  through  MAMS.  One  reason  given  by 
the  MAMS  users  was  that  many  FAA  eenters  are  not  using  SAMS,  whieh  is 
needed  for  FAA  to  inter faee  with  MAMS.  Another  reason  was  that  MAMS 
expeets  to  reeeive  usage  data  from  the  seheduler.  MAMS  eounts  all  seheduled 
events  as  used  unless  users  reopen  the  program  and  eheek  eaeh  speeifie  event  as 
“not  used.  ”  An  example  of  problems  relating  to  reporting  use  of  airspaee  is 
when  seheduling  ageneies  use  military  operating  areas  and  military  training 
routes.  The  seheduling  ageney  has  no  knowledge  whether  the  airspaee  was 
aetually  used,  whieh  results  in  inaeeurate  usage  reporting  if  someone  does  not 
report  in  MAMS  whether  eaeh  seheduled  event  oeeurred. 

Guidance  on  the  Use  of  MAMS.  Guidance  from  the  Air  Force  Director 
of  Operations  and  Training,  Deputy  Chief  of  Staff,  Air  and  Space  Operations 
and  the  Director,  Aerospace  Operations  for  the  Air  Combat  Command  further 
illustrated  Service  frustration  with  MAMS.  The  Director  issued  interim  e-mail 
guidance,  “Interim  Military  Airspace  Management  System  (MAMS)  Guidance,” 
to  the  Air  Force  on  February  6,  2001.  The  message  states  that  the  MAMS 
problems  are  more  than  simply  “growing  pains”  and  addresses  issues  with  the 
alert  system  and  system  access.  The  Director’s  guidance  further  states  that, 
while  the  developers  are  correcting  the  problems,  it  is  imperative  that  the  Air 
Force  continue  to  maintain  positive  control  over  the  scheduling  of  military 
airspace.  The  Director  tasked  the  units  to  comply  with  the  following  guidance: 

•  until  MAMS  reliability  improves,  units  will  continue  to  manually 
schedule  and  deconfiict  missions; 

•  once  manually  scheduled  and  deconflicted,  the  data  will  be  entered 
into  MAMS  to  exercise  the  system  and  capture  airspace  usage  data; 
and 

•  where  airspace  overlaps,  ensure  that  Letters  of  Agreement  provide 
guidelines  for  deconfliction  and  schedule  procedures  to  ensure  flight 
safety.  Units  scheduling  airspace  managed  by  other  Services  will 
continue  per  established  procedures  or  Letters  of  Agreement. 

Additionally,  the  Director,  Aerospace  Operations  in  a  memorandum  dated 
September  18,  2000,  to  Air  Combat  Command  Wing  Commanders,  states  that 
the  Air  Force  Associate  Directorate  for  Ranges  and  Airspace  directed  that  all 
Air  Combat  Command  units  integrate  MAMS  into  their  scheduling  process. 

The  unit  implementation  of  MAMS  was  to  commence  on  October  1,  2000. 
Despite  the  direction  from  the  Air  Force  Associate  Directorate  for  Ranges  and 
Airspace,  many  units  displayed  a  lack  of  interest  in  participating  in  the 
development  phase  and  have  opted  not  to  use  MAMS.  One  reason  was  that 
there  was  not  a  knowledgeable  airspace  manager  directly  tied  to  the  program.  It 
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is  imperative  that  experieneed  airspaee  managers  be  made  an  integral  part  of  the 
development  and  testing  phases.  As  a  result,  MAMS  is  not  the  single  eleetronie 
interfaee  for  providing  SUA  sehedules  and  historieal  aetivation  and  usage  data 
as  envisioned  in  the  Coneept  of  Operations,  whieh  diseusses  the  two  eategories 
of  interfaees,  the  other  DoD  seheduling  systems,  and  the  FAA  SAMS. 

User  Feedback  on  Survey  Questionnaire.  Like  the  user  interviews  and 
Serviee  doeumentation,  responses  to  the  survey  questionnaire  showed  that 
aeeount  holders  (users)  question  the  use  of  MAMS  and  are  not  satisfied  with  the 
system.  Speeifieally,  the  survey  showed  that  while  MAMS  was  highly 
aeeessible,  the  user  eommunity  remained  fundamentally  dissatisfied  with  the 
system.  Table  1  and  Table  2  indieate  the  pereentage  of  “yes”  responses 
regarding  MAMS  aeeessibility,  use,  and  satisfaetion. 


Table  1.  Percentage  of  Respondents  Answering  “Yes”  to 
Questions  Regarding  MAMS  Accessibility 


Ouestion 

Percentage 

Has  heard  of  MAMS  (191  of  193) 

99 

Has  internet  access  (192  of  193) 

99 

Has  MAMS  account  (177  of  193) 

92 

Note:  The  193  respondents  may  not  be  representative  of  the  621  current 
account  holders. 


Table  2.  Percentage  of  Respondents  Answering  “Yes”  to 
Questions  Regarding  MAMS  Use  and  Satisfaction 

Question 

Percentage 

Uses  MAMS  (50  of  185) 

27 

MAMS  has  required  functionality  (54  of  159) 

34 

MAMS  is  user  friendly  (58  of  158) 

37 

Recommends  MAMS  (61  of  165) 

37 

Note:  Percentages  reflect  actual  responses  to  each  question  and  may  not 
represent  the  193  surveys  returned  or  the  621  account  holders  surveyed. 
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Table  2  shows  that  only  50  of  185,  or  27  percent,  of  user  responses  indicated 
that  they  use  MAMS  to  schedule  SUA.  The  reasons  that  account  holders 
reported  low  usage  include:  not  all  units  are  scheduling  through  MAMS; 

MAMS  does  not  meet  needs  to  schedule  surface  and  subsurface  vessels;  there  is 
not  a  mandate  to  use  the  system;  and  it  is  too  slow,  difficult  to  use,  and  lacks 
effectiveness.  In  addition,  only  54  of  159,  or  34  percent,  of  respondents 
believed  that  MAMS  had  the  necessary  functionality.  Some  of  the  explanations 
respondents  provided  include  that  MAMS  is  too  time-consuming,  lacks  real-time 
scheduling  capability,  and  does  not  properly  address  Service-unique 
requirements.  User  feedback  concerning  a  low  level  of  user  friendliness  and 
desire  to  recommend  MAMS  frequently  noted  that  the  system  is  too  complicated 
for  everyday  use  without  extensive  training,  the  system  requires  too  much  data 
input,  and  that  too  many  screens  and  unnecessary  actions  are  required. 

In  addition,  survey  respondents  were  asked  to  indicate  which  scheduling  systems 
other  than  MAMS  they  were  using.  Of  the  128  respondents,  72  account  holders 
stated  that  they  schedule  manually;  17  used  legacy  systems  that  included  the 
Range  Facility  Management  Support  System,  the  Resource  Scheduling  and 
Operations  Management  System,  the  Military  Airspace  Management  System 
(MASMS),  which  was  the  predecessor  system  to  MAMS  that  had  the  same 
name  but  different  acronym;  and  the  Fleet  Area  Control  and  Surveillance 
Scheduling  System.  Another  39  account  holders  met  their  scheduling  needs 
through  other  methods,  including  locally  produced  Excel  spreadsheets, 
facsimile,  e-mail,  and  telephone. 

Interfacing  with  Other  DoD  Scheduling  Systems  and  FAA.  Two  categories 
of  external  interfaces  include  other  DoD  scheduling  systems  and  the  FAA 
SAMS.  MAMS  is  to  provide  a  single  generalized  interface  for  all  other  DoD 
scheduling  systems  and  the  FAA  SAMS.  Other  DoD  scheduling  systems  have 
not  yet  interfaced  with  MAMS.  Currently,  MAMS  interfaces  with  FAA; 
however,  FAA  received  less  than  1  percent  of  its  scheduled  messages  from 
MAMS  from  August  8,  2001,  through  January  30,  2002. 

MAMS  Interface  with  Other  DoD  Scheduling  Systems.  On 

January  29,  1999,  the  Services  and  the  Executive  Director  for  the  Policy  Board 
on  Federal  Aviation  signed  an  interface  agreement  that  provided  the  baseline 
description  for  the  exchange  of  information  between  the  DoD  MAMS  and  other 
DoD  scheduling  systems  concerning  SUA.  However,  the  agreement  was  neither 
intended  nor  designed  to  change  operational  policies,  existing  Letters  of 
Agreement  and  Letters  of  Procedures,  and  other  local  agreements  defining  the 
day-to-day  operation  of  the  National  Airspace  System,  nor  did  it  require  that  the 
Services  replace  the  existing  systems  for  scheduling  SUA  with  MAMS.  The 
interface  agreement  states  that  the  Services  have  the  option  to  use  MAMS, 
continue  with  the  existing  system  and  develop  an  interface  with  MAMS,  or  use 
a  combination  of  their  system  supplemented  by  direct  use  of  MAMS. 
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The  requirement  for  MAMS  to  interfaee  with  other  DoD  seheduling  systems 
was  still  not  operational  beeause  no  interfaee  existed  from  other  DoD  seheduling 
systems  to  MAMS.  The  Joint  Program  Offiee  developed  MAMS  to  be  a  system 
that  initially  interfaeed  with  other  DoD  SUA  seheduling  systems,  and  it  was 
intended  to  provide  a  eonsolidated  database  of  seheduling,  a  seheduling 
eapability  for  nonautomated  users,  and  an  eleetronie  interfaee  to  FAA  for  the 
transmission  of  eleetronie  sehedule  and  usage  data. 

MAMS  Interface  with  FAA.  On  Mareh  2,  1998,  the  Exeeutive 
Direetor  of  the  Poliey  Board  on  Federal  Aviation  and  the  FAA  Program 
Direetor  for  Air  Traffie  Operations  signed  “The  MAMS/SAMS  Interfaee 
Agreement  between  the  DoD  and  FAA.”  The  intent  of  the  interfaee  agreement 
was  for  the  DoD  airspaee-seheduling  ageneies  to  transmit  eleetronieally  an 
airspaee  sehedule  message  from  the  MAMS  Central  Faeility  direetly  to  the  FAA 
SAMS.  However,  the  majority  of  FAA  eenters  were  not  using  SAMS,  whieh 
was  designed  to  interfaee  with  MAMS. 

FAA  Centers.  The  FAA  had  26  air  eontrol  eenters  (23  Air 
Route  Traffie  Control  Center  and  3  towers).  Of  the  26  eenters,  10  frequently 
use  SAMS,  6  eenters  were  not  frequently  using  SAMS,  and  10  sites  were  unsure 
of  their  status  with  SAMS. 

MAMS  and  SAMS  Usage.  The  FAA  manually  seheduled 
about  500  to  800  sehedules  per  day.  Data  reeeived  from  an  FAA  representative 
diselosed  that  from  August  8,  2001,  through  January  30,  2002,  SAMS  reeeived 
540  messages  from  MAMS.  Comparing  the  volume  of  MAMS  seheduled 
messages  to  SAMS  during  the  same  period  to  manually  seheduled  entries  into 
SAMS  per  month  showed  that  MAMS  messages  were  less  than  1  pereent  of  the 
total  volume. 


MAMS  Information  Exchange  with  FAA  SAMS.  As  defined 
in  the  DoD  FAA  Interageney  Agreement  on  the  MAMS/SAMS  interfaee,  dated 
Mareh  2,  1998,  MAMS  will  provide  sehedule  and  usage  data  to  SAMS.  SAMS 
sehedule  data  inelude  the  unit  site,  airspaee  name,  date,  start  time,  low  entry 
point,  exit  time,  and  high  exit  point.  Prior  to  the  full  deployment  of  the  FAA 
SAMS,  sehedule  information  will  be  forwarded  in  aeeordanee  with  existing 
proeedures.  Onee  MAMS  is  deployed,  the  users  will  provide  eleetronie 
sehedules  to  FAA.  The  interfaee  eontrol  doeument  between  the  DoD  MAMS 
and  the  FAA  SAMS,  signed  April  16,  1999,  provides  detailed  protoeols  and 
formats  for  the  exehange  of  information. 

DoD  Sponsorship  and  Fnnding.  The  MAMS  program  laeked  a  DoD 
funetional  sponsor  and  adequate  funding  to  address  known  system  problems, 
despite  the  effort  and  motivation  of  the  MAMS  Central  Faeility  to  improve 
MAMS  aeeeptanee. 

DoD  Sponsorship.  MAMS  laeked  a  DoD  funetional  sponsor  and  did 
not  have  any  near-term,  future  programmed  funding  for  major  improvements 
beeause  MAMS  is  an  Air  Foree-managed  program  that  did  not  have  authority  to 
program  funding.  MAMS  was  also  an  Aequisition  Category  III  program; 
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therefore,  funding  support  was  a  lower  priority  within  the  Air  Foree  and  other 
Serviees.  Further,  the  Air  Foree  Flight  Standards  Ageney  must  fund  system 
modifieations,  and  any  proposed  MAMS  modifieation  must  eompete  for  funding 
with  other  Air  Traffie  Control  and  Landing  Systems  for  3080  (Other 
Proeurement)  money. 

System  Funding.  The  Air  Foree  eost  for  developing  MAMS  from 
Mareh  1990  through  September  2000  was  $24.4  million.  The  Army,  Navy,  and 
Air  Foree  provided  funding  of  $2  million  per  year  for  operating  and  maintaining 
MAMS  sinee  it  was  fielded  in  Oetober  2000.  The  sustainment  funding  was 
prorated  as  follows:  the  Navy  and  Air  Foree  eaeh  eontributed  $800,000 
(40  pereent)  and  the  Army  eontributed  $400,000  (20  pereent).  The  Air  Foree 
also  projeeted  an  annual  sustainment  budget  of  $2  million  in  future  years.  The 
MAMS  projeet  management  initially  eonsidered  a  major  teehnieal  refreshment 
for  MAMS  from  FY  2005  through  FY  2007,  with  total  estimated  eosts  of 
$13  million,  but  later  withdrew  its  eonsideration.  The  teehnieal  refreshment 
eonsidered  by  the  projeet  management  offiee  would  have  ineluded  revisions  to 
the  Digital  Aeronautieal  Flight  Information  File  and  proeurement  of  additional 
eomputer  and  server  teehnology  for  the  MAMS  Central  Faeility.  The  Air  Foree 
cannot  sustain  any  future  technical  refreshment  with  the  current  yearly 
commitment  of  $2  million  in  sustainment  funding. 

Testing  Requirements.  Although  the  Air  Force  fielded  MAMS  in  October 
2000,  it  had  not  tested  all  required  elements  of  the  Requirements  Underlying 
Baseline  for  MAMS,  including  the  MAMS  load  balancing  test,  the  Sybase 
Replication  Server  test,  and  other  DoD  scheduling  system  tests.  Those  tests 
were  needed  to  ensure  MAMS  availability,  reliability,  and  maintainability  within 
the  user  community.  Other  significant  testing  limitations  included  a  waiver  for 
Operational  Test  and  Evaluation  for  MAMS  and  limitations  to  subsequent  test 
and  evaluation  and  acceptance  tests. 

Waiver  for  Operational  Test  and  Evaluation.  On  November  17, 
1997,  the  Air  Force  Operational  Test  and  Evaluation  Center  issued  a 
memorandum,  “Operational  Test  and  Evaluation  Determination  for  Military 
Airspace  Management  System  (MAMS),”  stating  that  an  analysis  of  the  test 
strategy  for  the  MAMS  was  completed  from  October  1  through  October  2, 

1997.  The  Air  Force  Test  and  Evaluation  Center  waived  the  operational  test 
and  evaluation  for  MAMS  because  the  results  of  the  analysis  determined  that  the 
initial  development  for  a  stand-alone  hardware  system  using  newly  developed 
software  compatible  with  various  DoD  systems  had  changed  to  an  Internet-based 
software  system,  resulting  in  low-risk  development. 

Combined  Test  and  Evaluation.  The  Combined  Test  and  Evaluation 
for  MAMS  was  conducted  in  FY  1998  and  the  Delta  (Spiral)  Combined  Test 
and  Evaluation  was  conducted  in  FY  1999.  The  purpose  of  the  tests  was  to 
identify  the  extent  that  MAMS  was  able  to  meet  testing  objectives.  Similar 
limitations  existed  in  both  tests,  such  as  two  limitations  that  related  to  the 
external  systems  not  fully  implementing  required  interfaces  at  test  time,  and  one 
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limitation  regarding  MAMS  program  implementation  delay,  whieh  delayed  the 
Defense  Information  Infrastrueture  Common  Operating  Environment  eomplianee 
testing. 


MAMS  Acceptance  Tests.  The  Eleetronie  System  Center,  Hanseom 
Air  Eoree  Base,  Massaehusetts  tasked  the  46th  Test  Squadron,  Eglin  Air  Eoree 
Base,  Elorida,  to  eonduet  a  teehnieal  evaluation  of  the  MAMS.  The  teehnieal 
evaluation  ineluded  the  results  of  the  aeeeptanee  test  on  the  system  as  eonfigured 
for  the  MAMS  Central  Eaeility.  The  system  was  tested  by  the  eontraetor, 
Raytheon,  using  proeedures  approved  by  the  Eleetronie  Systems  Center  MAMS 
program  offiee.  Government  and  eontraetor  aeeeptanee  tests  had  signifieant 
limitations. 


Government  Acceptance  Testing.  The  Air  Eoree  Eleetronie 
Systems  Center,  Hanseom  Air  Eoree  Base,  Massaehusetts,  tasked  the  46th  Test 
Squadron,  the  responsible  test  organization,  to  eonduet  aeeeptanee  testing  on  the 
MAMS  version  2.1.  The  testing  was  eondueted  from  Eebruary  29  through 
Mareh  22,  2000.  The  purpose  of  the  aeeeptanee  testing  was  to  verify  the 
Requirements  Underlying  Baseline,  whieh  was  ereated  to  expand  on  the 
requirements  in  the  MAMS  Joint  Operational  Requirements  Doeument.  The 
46th  Test  Squadron  did  not  test  the  MAMS  eapability  to  distribute  reports, 
verify  software  maintainability,  verify  hardware  maintainability,  verify 
meantime  between  eritieal  failure  of  individual  work  stations  requirements,  and 
demonstrate  fault  deteetion  eapability. 

Contractor  Acceptance  Testing.  Raytheon  eondueted 
funetional  tests  on  MAMS  versions  2.2.2  performanee  tests  from  Deeember  4 
through  Deeember  6,  2001.  Testing  eontinued  at  the  Defense  Information 
Systems  Ageney  from  Deeember  11  through  Deeember  14,  2001.  Test  results 
from  “MAMS  Central  Eaeility  Support  Test  Report  for  the  Military  Airspaee 
Management  System  (MAMS),”  version  2.2.2,  Eebruary  2002,  show  that  the 
eontraetor  did  not  eonduet  the  following  planned  tests: 

•  The  MAMS  Load  Balaneing  test  —  ensures  no  interruption  of 
serviee  if  the  primary  server  fails; 

•  The  Sybase  Replieation  Server  test  —  ensures  that  the  baekup 
database  server  has  no  interruption  of  serviee  if  the  primary 
database  fails; 

•  The  Digital  Aeronautieal  Elight  Information  Eile  test  —  an  aetion 
taken  by  a  user  to  update  the  Elight  Information  Publieation 
database  with  new  Digital  Aeronautieal  Elight  Information  Eile 
data; 

•  The  Save  and  Restore  Baseline  test  —  used  to  save  the  MAMS 
database  and  load/restore  it  on  a  server;  and 

•  The  Other  DoD  Seheduling  System  test  —  interfaee  test  was  not 
performed  beeause  the  operational  interfaee  was  not  available. 


13 


System  Vulnerability  Test.  In  May  2000,  the  Air  Force  Information 
Warfare  Center  performed  a  vulnerability  assessment  test  on  MAMS.  The 
results  of  the  vulnerability  assessment  found  numerous  system  vulnerabilities  to 
include  access  concerns  that  involved  user  identification  and  password.  For 
example,  the  Air  Force  Information  Warfare  Center  established  that  several  user 
accounts  did  not  fully  comply  with  the  Air  Force  password  policy.  Specifically, 
access  was  permitted  when  user  names  and  passwords  were  the  same,  and 
passwords  could  be  easily  cracked  because  of  noncompliance  with  a 
recommended  mix  of  upper  and  lowercase  letters,  numerals,  and  special 
characters.  As  of  April  2002,  the  MAMS  Central  Facility  had  not  corrected  all 
of  the  system  vulnerabilities  that  were  outlined  in  the  Air  Force  Information 
Warfare  Center  assessment. 

Conclusion 


After  10  years  of  development  and  fielding,  and  expenditures  of  $24  million,  the 
MAMS  had  not  met  its  original  objective  to  become  the  DoD-wide  system  for 
managing  military  airspace  scheduling  functions  and  providing  usage 
information  to  DoD  and  FAA.  Based  on  discussions  with  users.  Service 
documentation,  and  user  survey  results,  the  Assistant  Secretary  Defense 
(Command,  Control,  Communications,  and  Intelligence)  needs  to  reevaluate  the 
MAMS  program  and  the  Service  SUA  requirements  to  determine  whether  the 
MAMS  is  the  most  cost-effective  system  to  meet  future  DoD  requirements  for 
scheduling  SUA  and  usage  reporting.  The  MAMS  program  has  not  gained 
widespread  acceptance  and  use  among  the  Services,  despite  the  MAMS  Central 
Facility  dedication  and  progress  in  correcting  problems. 

Air  Force  Comments  on  the  Report  and  Audit  Response 


The  Deputy  Chief  of  Staff  for  Air  and  Space  Operations,  Department  of  the  Air 
Force  provided  unsolicited  comments  on  the  allegations.  Audit  results  are 
discussed  in  Appendix  B.  A  summary  of  Air  Force  comments  and  the  audit 
response  is  in  Appendix  F. 

Recommendations,  Management  Comments,  and  Audit 
Response 

A.  We  recommend  that  the  Assistant  Secretary  of  Defense  (Command, 
Control,  Commnnications,  and  Intelligence)  task  the  Air  Force,  as  the 
execntive  agent  for  the  Military  Airspace  Management  System,  to: 

1.  Establish  a  Military  Airspace  Management  System  in-process 
action  team,  which  inclndes  representation  at  the  nser  level  from  all 
Services,  to  condnct  an  analysis  of  the  Military  Airspace  Management 


14 


System  and  determine  whether  it  is  the  most  cost-effective  schednling 
program  to  meet  DoD  fntnre  airspace  schednling,  nsage,  and  reporting 
reqnirements.  The  in-process  action  team  shonld: 

a.  Identify  each  Service’s  fntnre  reqnirements  for  Special  Use 
Airspace  schednling,  nsage,  and  reporting; 

b.  Evalnate  any  new  technology  available  for  schednling  and 
reporting  nsage  data  of  Special  Use  Airspace  to  the  Federal  Aviation 
Administration;  and 

c.  Consider  whether  it  wonld  be  more  cost-effective  if  each  of 
the  Services  nses  its  cnrrent  schednling  system  and  develops  an  interface 
directly  with  the  Federal  Aviation  Administration. 

Assistant  Secretary  of  Defense  (Command,  Control,  Commnnications,  and 
Intelligence)  Comments.  The  Director,  Communication  Programs,  responding 
for  the  Assistant  Secretary  of  Defense  (Command,  Control,  Communications, 
and  Intelligence),  concurred. 

Department  of  the  Air  Force  Comments.  Although  not  required  to  comment, 
the  Deputy  Chief  of  Staff  for  Air  and  Space  Operations,  Department  of  the  Air 
Force  (the  Deputy  Chief  of  Staff)  stated  that  the  Air  Force  agreed  with  the 
recommendations  but  would  be  hesitant  to  agree  with  allowing  the  other 
Services  to  interface  directly  with  the  Federal  Aviation  Administration,  due  to 
safety  concerns.  The  Deputy  Chief  of  Staff  provided  an  estimated  completion 
date  of  July  2003  for  the  MAMS  in-process  action  team  evaluation. 

Audit  Response.  In  response  to  the  final  report,  we  request  that  the  Assistant 
Secretary  of  Defense  (Command,  Control,  Communications,  and  Intelligence) 
provide  an  action  plan  and  completion  dates  for  implementing  the 
recommendation.  We  agree  that  safety  concerns  should  be  considered  in 
deciding  whether  the  other  Services  should  interface  directly  with  the  Federal 
Aviation  Administration  in  scheduling  air  space. 

2.  Suspend  any  future  installation  of  upgrades  or  technical 
refreshment  efforts  for  the  Military  Airspace  Management  System  until 
completing  the  cost-effectiveness  analysis. 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications,  and 
Intelligence)  Comments.  The  Director,  Communication  Programs,  responding 
for  the  Assistant  Secretary  of  Defense  (Command,  Control,  Communications, 
and  Intelligence),  concurred,  stating  that  his  office  will  fund  only  the  MAMS 
operations  and  maintenance  costs  until  completion  of  the  cost-effectiveness 
analysis. 
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Department  of  the  Air  Force  Comments.  Although  not  required  to  eomment, 
the  Deputy  Chief  of  Staff  agreed  with  the  reeommendation  but  suggested  that 
minor  upgrades  and  teehnieal  refreshment  efforts  were  required  to  ensure  that 
the  MAMS  remains  operational. 

Audit  Response.  The  comments  from  the  Director,  Communication  Programs 
were  responsive.  We  recognize  that  the  Air  Force  may  need  to  make  minor 
expenditures  to  keep  the  MAMS  operational  until  the  Assistant  Secretary  of 
Defense  (Command,  Control,  Communications,  and  Intelligence)  and  the 
Services  determine  the  most  cost-effective  alternative  for  scheduling  airspace. 

B.  We  recommend  that  the  Assistant  Secretary  of  Defense  (Command, 
Control,  Communications,  and  Intelligence)  task  the  Army  Deputy  Chief  of 
Staff  for  Operations  and  Plans;  the  Head,  Airspace  and  Air  Traffic 
Control,  Department  of  the  Navy;  and  Director  of  Operations  and  Training, 
Deputy  Chief  of  Staff  for  Air  and  Space  Operations,  Headquarters  U.S.  Air 
Force  to  develop  and  implement  policies  and  procedures  to  standardize  the 
Special  Use  Airspace  scheduling  process. 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications,  and 
Intelligence)  Comments.  The  Director,  Communication  Programs,  responding 
for  the  Assistant  Secretary  of  Defense  (Command,  Control,  Communications, 
and  Intelligence),  concurred. 

Department  of  the  Air  Force  Comments.  Although  not  required  to  comment, 
the  Deputy  Chief  of  Staff  agreed  with  the  recommendation  stating  that  the  Air 
Force  is  already  in  compliance. 

Audit  Response.  In  response  to  the  final  report,  we  request  that  the  Assistant 
Secretary  of  Defense  (Command,  Control,  Communication,  and  Intelligence) 
provide  an  action  plan  and  completion  dates  for  implementing  the 
recommendation.  Although  the  Air  Force  does  have  policies  and  procedures 
requiring  a  standardized  Special  Use  Airspace  scheduling  process,  a 
standardized  process  has  not  been  fully  implemented  throughout  the  Air  Force. 
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Appendix  A.  Scope  and  Methodology 

Scope 


Work  Performed.  We  reviewed  doeumentation  dating  from  Mareh  1987 
through  April  2002  ineluding  baekground  information,  mission  and  operational 
need  statements,  training  and  operational  requirements,  and  eontraeting  and 
budget  doeuments.  Also,  we  eondueted  interviews  with  offieials  at  Tinker  Air 
Foree  Base,  Oklahoma;  Hanseom  Air  Foree  Base,  Massaehusetts;  FAA 
headquarters,  and  the  Defense  Enterprise  Computing  Center  to  gain  information 
on  MAMS  operations.  Further,  we  reviewed  193  responses  on  survey 
questionnaires  from  MAMS  users  in  all  Serviees  to  assess  user  satisfaetion  with 
the  MAMS  program.  Finally,  we  attended  the  November  15,  2001, 
Configuration  Control  Board  meeting  at  Fort  Bel  voir,  Virginia,  and  visited 
several  sites  within  the  Serviees  that  were  seheduling  Speeial  Use  Airspaee. 

Limitation  to  Scope.  Beeause  our  objeetive  was  limited  to  evaluating  a 
Defense  Hotline  allegation  related  to  the  effieieney  and  effeetiveness  of  the 
Military  Airspaee  Management  System,  its  eapabilities,  limitations,  and  training 
requirements,  we  did  not  review  the  management  eontrol  program. 

General  Acconnting  Office  High-Risk  Area.  The  General  Aeeounting  Offiee 
has  identified  several  high-risk  areas  in  the  DoD.  This  report  provides  eoverage 
of  the  DoD  System  Modernization  high-risk  area. 

Methodology 


To  assess  allegations  eoneerning  the  effieieney  and  effeetiveness  of  the  MAMS, 
we  developed  a  survey  questionnaire,  whieh  we  sent  to  621  aetive  aeeount 
holders  within  the  Departments  of  the  Army,  Navy,  and  Air  Foree.  To 
maximize  survey  partieipation  and  to  eneourage  eandid  responses,  we  promised 
respondents  to  eaeh  survey  questionnaire  that  we  would  keep  their  individual 
responses  eonfidential. 

Use  of  Computer-Processed  Data.  We  did  not  rely  on  eomputer-proeessed 
data  or  statistieal  proeedures. 

Use  of  Technical  Assistance.  Personnel  from  the  Audit  Follow-up  and 
Teehnieal  Support  Direetorate  assisted  us  during  the  audit.  The  Quantitative 
Methods  Division  assisted  in  the  development  and  analysis  of  survey  results. 
The  Teehnieal  Assessment  Division  assisted  in  reviewing  test  plans  and  results. 

Audit  Type,  Dates,  and  Standards.  We  eondueted  this  eeonomy  and 
effieieney  audit  from  September  2001  through  April  2002  in  aeeordanee  with 
generally  aeeepted  government  auditing  standards. 
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Contacts  During  the  Audit.  We  visited  or  eontaeted  individuals  and 
organizations  within  DoD,  seleeted  DoD  eontraetors,  and  the  Department  of 
Transportation.  Further  details  are  available  upon  request. 

Prior  Coverage 

During  the  past  5  years,  there  have  been  no  reports  issued  eoneerning  the 
Military  Airspaee  Management  System. 
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Appendix  B.  Allegations  and  Audit  Results 


On  July  5,  2001,  a  complainant  submitted  seven  allegations  to  the  DoD  Hotline. 
Those  allegations  expressed  eoneerns  about  the  effieieney  and  effeetiveness  of 
the  Military  Airspaee  Management  System  (MAMS)  in  several  areas.  Of  the 
seven  allegations,  we  substantiated  four,  partially  substantiated  two  and  did  not 
substantiate  one. 

Allegation  1:  Training  for  the  system  is  insnfficient. 

Partially  Snbstantiated.  We  determined  that  MAMS  training  was  adversely 
affeeted  beeause  of  limited  funding  and  resourees,  laek  of  eoordination  within 
the  user  eommunity,  and  diseord  between  the  evolving  MAMS  system  and  the 
available  training  materials. 

The  MAMS  Central  Faeility  stopped  paying  for  travel  expenses  assoeiated  with 
the  training,  and  the  Navy  stated  that  it  had  not  budgeted  for  MAMS  training 
eosts. 

Many  users  did  not  have  the  neeessary  MAMS  equipment  when  they  reeeived 
training,  or  familiarity  and  expertise  with  the  system.  If  they  had  the  equipment, 
it  was  not  up  and  running.  As  a  result,  too  mueh  time  lapsed  between  training 
and  use. 

Training  materials  ehanged  with  eaeh  version  upgrade,  eausing  additional 
problems  with  eost  and  notifieation  to  the  users  for  updated  materials. 

Currently,  the  MAMS  Central  Faeility  has  limited  resourees  for  training.  The 
$2  million  sustainment  fund  ineluded  training,  manuals,  and  travel  for 
instruetors. 

Further,  user  eomments  on  the  survey  related  to  training  noted  that: 

•  there  is  not  enough  time  in  elass  to  learn  a  eomplex  program  like 
MAMS  (13  aeeount  holders); 

•  the  training  was  not  well-organized  (9  aeeount  holders); 

•  the  training  elass  is  time-eonsuming  and  diffieult  to  understand 
(4  aeeount  holders). 

•  beeause  of  limited  resourees,  the  Serviees  do  not  have  the  money 
available  for  training  (1  aeeount  holder);  and 

•  the  user  manual  is  rarely  refereneed  or  used  during  the  training 
(1  aeeount  holder). 
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Allegation  2:  Services  and  schednling  agencies  are  not  nsing  the  system. 

Snbstantiated.  User  response  to  the  survey  questionnaire  substantiated  that  the 
Serviees  and  seheduling  ageneies  were  not  using  MAMS.  Speeifieally,  out  of 
193  survey  responses  reeeived,  128  aeeount  holders  (5  Army,  11  Navy,  105  Air 
Foree,  and  7  Marine  Corps)  stated  that  they  were  using  something  other  than 
MAMS  or  in  addition  to  MAMS.  See  Appendix  D  for  details.  Additionally, 
during  the  audit,  an  Air  Foree  user  stated  that  although  his  organization  sent 
members  of  the  staff  for  training,  it  made  no  sense  for  them  to  attempt  to  use  a 
system  that  no  one  else  was  using.  Also,  users  noted  that  most  units  had  no 
time  or  manpower  to  dedieate  to  MAMS  and  that  their  eurrent  seheduling 
system  was  working  effieiently  and  effeetively.  We  did  not  meet  with  any  units 
that  were  using  the  system  to  its  full  potential. 

Allegation  3:  The  program  is  being  asked  to  do  too  many  fnnctions  for  too 
many  different  types  of  airspaces. 

Snbstantiated.  User  response  to  the  survey  questionnaire  substantiated  that  the 
system  was  being  asked  to  do  too  many  funetions  to  satisfy  the  diversity  of 
airspaee  and  the  Serviee  requirements.  Of  the  159  survey  respondents  who 
answered  this  question,  105,  or  66  pereent,  stated  that  MAMS  did  not  have  the 
funetionality  for  them  to  do  their  job. 

Also,  survey  respondent  eomments  indieated  that  using  MAMS  ean  greatly 
inerease  workload  beeause  of  the  way  MAMS  views  airspaee.  For  example,  an 
airspaee  ean  be  divided  into  seetions  that  are  labeled  A,  B,  C,  D,  and  E  and 
those  seetions  are  designed  to  aetivate  together  for  training  where  aireraft 
approaeh  at  different  altitudes.  The  Air  Foree  seheduling  system  sehedules  the 
subdivisions  as  one  entity;  however,  MAMS  ealls  those  subdivisions  Dynamie 
airspaee  and  identifies  them  as  different  airspaees.  MAMS  requires  the 
requester  to  aetivate  and  deaetivate  the  five  different  airspaees  (A  through  E) 
independently  of  one  another.  Also,  a  eonfliet  that  shows  up  in  MAMS  would 
require  users  to  make  five  resolutions,  rather  than  one. 

In  addition,  users  stated  that: 

•  MAMS  does  not  resolve  eonfiiets  with  other  SUAs  and  military 
training  routes  (22  aeeount  holders); 

•  MAMS  eauses  a  duplieation  of  efforts,  it  is  too  labor  intensive, 
Internet  dependent,  and  redundant  (21  aeeount  holders);  and 

•  MAMS  does  not  have  the  flexibility  of  short-term  seheduling  and  it  is 
not  mandated  for  use  (5  aeeount  holders). 

Allegation  4:  Many  of  the  nsers  do  not  have  access  to  the  proper  compnter 
eqnipment  with  Internet  access  (MAMS  is  an  Internet-based  system). 

Unsnbstantiated.  We  did  not  substantiate  that  signifieant  numbers  of  users 
laeked  aeeess  to  the  proper  equipment  with  Internet  aeeess,  as  only  one  Army 
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response  expressed  eoneern  with  equipment  and  Internet  aeeess  issues.  While 
we  did  not  evaluate  whether  users  have  aeeess  to  the  proper  eomputer 
equipment,  aeeount  holder  response  to  the  survey  questionnaire  indieated  that 
almost  all  users  had  the  Internet  aeeess  needed  to  use  MAMS.  Speeifieally, 

192  of  the  193  survey  responses  stated  that  they  had  Internet  aeeess. 

Allegation  5:  The  time,  money  and  effort  utilized  in  building  and  training 
for  the  program  is  a  waste. 

Partially  Substantiated.  While  not  a  eomplete  waste,  the  money  that  the  Air 
Foree  spent  on  the  MAMS  was  not  effeetive  beeause  the  system  still  had  not  met 
many  Serviee  requirements  when  the  Joint  Program  Offiee  fielded  MAMS  in 
Oetober  2000,  after  10  years  of  development.  Speeifieally,  of  the 
159  respondents  who  answered  this  question,  105,  or  66  pereent,  stated  that 
MAMS  did  not  have  all  the  funetionality  needed  for  airspaee  management. 

Sinee  MAMS  fielding,  the  eontraetor  had  issued  several  versions  of  MAMS 
under  the  eurrent  eontraet  to  enhanee  the  system’s  funetionality.  However,  the 
Serviees,  partieularly  the  Army  and  the  Navy,  were  not  fully  using  the  system 
beeause  it  did  not  meet  Service-unique  requirements  for  the  diversity  of  SUA. 

Allegation  6:  The  scheduling  process  through  the  system  is  cumbersome 
and  is  time-consuming  compared  to  the  procedures  already  established. 

Substantiated.  The  Services  stated  that  their  current  scheduling  systems  work 
well  and  are  easier  to  use  than  MAMS.  As  discussed  in  the  Finding  section, 
users  stated  that  in  MAMS,  it  takes  12-15  minutes  to  log  on  the  Internet  site  and 
that  after  logging  on,  there  are  13  steps  they  must  complete  to  request  a  mission. 
Once  they  have  entered  the  request,  users  stated  that  they  have  to  wait  for 
someone  on  the  scheduling  end  to  send  a  response  to  accept  or  deny  the  mission. 
If  the  scheduling  unit  is  not  using  MAMS,  or  does  not  use  it  on  a  full-time 
basis,  the  request  may  not  be  approved  in  an  acceptable  amount  of  time. 
Conversely,  with  current  scheduling  systems,  the  requester  calls  the  scheduler 
and  the  scheduler  enters  the  data  into  an  Excel  spreadsheet  in  order  to  make  the 
decision  whether  to  accept  to  deny  the  mission  just  two  or  three  steps.  Out  of 

193  survey  respondents,  100,  or  63  percent,  stated  that  MAMS  is  not  user 
friendly.  Major  comments  included  that  the  program  is  labor  intensive,  very 
slow  even  on  powerful  computers,  and  the  interface  is  cumbersome  and  too 
complicated  for  everyday  use  without  extensive  training.  Also,  there  is  too 
much  duplication  of  effort  to  do  a  simple  task. 

Allegation  7:  The  utilization  reports  generated  from  the  MAMS  program 
are  not  sufficient  to  use  for  the  reports  that  we  send  to  other  agencies 
requiring  similar  but  more  detailed  information. 

Substantiated.  In  addition  to  the  Services’  requirement  to  report  usage  of 
SUA,  they  are  required  to  provide  other  reports  within  their  organization  and  to 
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other  agencies,  such  as  reporting  to  the  Environmental  Protection  Agency  on 
clean  up  of  hazardous  materiel  used  during  training  or  strategic  missions.  The 
MAMS  did  not  provide  the  tracking  or  the  level  of  detail  needed  for  these 
reporting  requirements.  Also,  the  majority  of  FAA  centers  were  not  using 
SAMS,  which  the  FAA  needs  to  interface  with  MAMS;  therefore,  many  of  the 
DoD  users  felt  no  need  to  use  MAMS. 
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Appendix  C.  MAMS  User  Survey  Questionnaire 


Name _  _ Requester 

_ Army _ Navy _ AF _ MC  _ Scheduler 

Unit _  _ Airspace  Manager 

If  applicable, _ Nat’l  Guard _ Reserve  _ Super  Requester 

MAMS  Questionnaire 

1)  Have  you  heard  of  the  Military  Airspace  Management  System  (MAMS)?  _ Yes _ No 

2)  Do  you  have  access  to  the  Internet? _ Yes _ No 

3)  Do  you  have  a  MAMS  account? _ Yes _ No  If  yes,  on  average,  how  often  on  a  monthly 

basis  do  you  access  the  system? 

4)  Have  you  received  MAMS  training? _ Yes _ No  If  yes,  when,  where,  and  how  long? 

5)  Do  you  feel  training  for  the  system  was  adequate? _ Yes _ N/A _ No  If  no,  please  explain. 

6)  If  you  are  an  Airspace  Manager,  have  you  verified  the  accuracy  of  your  airspace  data  in 

MAMS?  _ Yes _ No  If  no,  please  explain  why. 

7)  Are  you  currently  using  the  Military  Airspace  Management  System  (MAMS),  which  is  a 

centralized  database,  to  schedule  special  use  airspace  (SUA)?  _ Yes  _ No  If  no,  please 

explain  why. 

8)  Does  MAMS  have  the  functionality  necessary  for  you  to  do  your  job?  _ Yes _ No  If  no, 

please  explain. 

9)  If  not  using  MAMS  to  schedule  special  use  airspace,  are  you  currently  using  (please  check  all 
that  apply): 

_ RFMSS  _ RESOMS 

_ MASMS  _ FACSKED 

_ OTHER  _ Manual  scheduling  methods? 

If  you  are  using  OTHER,  what  type  of  program  are  you  using? 

10)  Do  you  find  MAMS  user  friendly? _ Yes _ No  Explain  your  answer. 

11)  Do  you  recommend  using  MAMS? _ Yes _ No  If  no,  why  not?  Please  provide  positive  and 

negative  comments. 
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Appendix  D.  Responses  to  MAMS  User  Survey 

Questionnaire 


Questionnaire  Responses  From  Miiitary  Services 

Q1 
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Q7 

Uses 

MAMS 

Q8 

Does  MAMS 
have 

functionaiity 
for  you  to  do 
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Appendix  E.  Service’s  Existing  Scheduling 

Systems 


Some  scheduling  systems  in  use  by  the  Services  include; 

FACSKED.  The  Navy’s  Fleet  Area  Control  and  Surveillance  Facilities  uses  the 
FACSKED  system.  The  Concept  of  Operations  states  that  FACSKED  will 
eventually  be  replaced  by  MAMS;  however,  the  Navy  stated  that  FACSKED  is 
being  replaced  by  the  Navy  Scheduling  System. 

Military  Airspace  Management  System.  The  Military  Airspace  Management 
System  (MASMS),  which  was  predecessor  to  MAMS  with  the  same  name  but 
different  acronym,  was  an  Air  Combat  Command-managed  system.  The 
Department  of  the  Air  Force  issued  a  message  on  July  20,  1999,  instructing  all 
Air  Force  units  to  integrate  MAMS  version  2.0. 

Resource  Scheduling  and  Operations  Management  System.  The  Resource 
Scheduling  and  Operations  Management  System  is  the  computerized  system 
established  at  Eglin  Air  Force  Base,  Florida,  for  scheduling  missions  requiring 
the  use  of  Air  Force  Development  Test  Center  ranges  or  resources.  The  system 
is  used  to  submit  mission  requests,  requests  for  munitions  support,  and  approved 
profiles  (safety,  flight,  or  ground)  for  specified  missions. 

Range  Facility  Management  Support  System.  The  Army  developed  the 
Range  Facility  Management  Support  System  and  the  Army  and  Marine  Corps 
use  the  system  to  schedule  air  and  ground  resources  and  operations.  The 
contractor  performed  a  limited  evaluation  of  the  MAMS  and  Range  Facility 
Management  Support  System  interface.  The  schedule  for  Range  Facility 
Management  Support  System  to  develop  a  MAMS  interface  is  from  FY  2003  to 
FY  2004,  although  full  airspace  scheduling  functionality  may  not  be  available  by 
that  date  under  the  current  funding  plan. 

Tactical  Aircrew  Scheduling  and  Airspace  Management  System.  The  Air 

Force  Tactical  Aircrew  Scheduling  and  Airspace  Management  System  is  a  real¬ 
time  scheduling  program  that  the  Air  Force  Wings  use.  It  is  the  primary  tool 
used  to  communicate  operational  requirements  to  the  various  support  agencies. 
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Appendix  F.  Audit  Response  to  Air  Force 

Comments  on  the  Report 


Our  detailed  responses  to  the  unsolieited  eomments  that  the  Deputy  Chief  of 
Staff  for  Air  and  Spaee  Operations,  Headquarters,  U.S.  Air  Foree  (the  Deputy 
Chief  of  Staff)  provided  on  Appendix  B,  Allegations  and  Audit  Results,  follow. 
The  eomplete  text  of  the  management  eomments  on  statements  in  the  draft 
report  is  in  the  Management  Comments  seetion  of  this  report. 

Allegation  1:  Training  for  the  system  is  insnfficient  (partially 
snbstantiated). 

Management  Comments.  The  Deputy  Chief  of  Staff  agreed,  stating  that  the 
MAMS  Central  Faeility  at  Tinker  Air  Foree  Base  reeognized  that  MAMS 
training  was  not  adequate  and  took  steps  to  eorreet  the  defieieney.  Speeifieally, 
the  Deputy  Chief  of  Staff  stated  that  he  ensured  that  students  had  a  valid  MAMS 
aeeount,  knowledge  of  the  MAMS  user  manual,  and  knowledge  of  their  unit’s 
airspaee  seheduling  proeedures  before  attending  training,  and  that  he  has 
modified  eourse  eontent  and  handouts.  Further,  the  Deputy  Chief  of  Staff  stated 
that  Serviee  units  will  eontinue  to  fund  training  eosts  for  MAMS  students  until 
the  Joint  Program  Offiee  ean  provide  funding  at  no  eost  to  the  user. 

Audit  Response.  The  Air  Foree  has  taken  aetions  that  will  improve  MAMS 
training. 

Allegation  2.  Services  and  scheduling  agencies  are  not  using  the  system 
(substantiated). 

Management  Comments.  The  Deputy  Chief  of  Staff  agreed,  stating  that  the 
Air  Foree  units  were  not  fully  using  the  system  beeause  the  MAMS  tool  for 
deteeting  seheduling  eonfliets  was  malfunetioning.  The  Deputy  Chief  of  Staff 
also  stated  that  the  Air  Foree  was  modifying  existing  Air  Foree  airspaee 
seheduling  systems  to  use  the  Other  DoD  Seheduling  Systems  interfaee  and  that, 
onee  interfaees  are  eomplete,  MAMS  will  extraet  usage  data.  Additionally,  the 
Deputy  Chief  of  Staff  stated  that  a  followup  report  is  due  September  30,  2002. 

Allegation  3.  The  program  is  being  asked  to  do  too  many  functions  for  too 
many  different  types  of  airspace  (substantiated). 

Management  Comments.  The  Deputy  Chief  of  Staff  disagreed,  stating  that 
MAMS  provides  users  with  the  eapability  to  effieiently  sehedule  airspaee  and 
aeeurately  doeument  and  report  usage  to  the  FAA,  as  well  as  the  eapability  for 
identifying  eonfliets.  In  addition,  the  Deputy  Chief  of  Staff  stated  that  the  Air 
Foree  made  numerous  improvements  to  MAMS  in  response  to  user  requests, 
and  that  MAMS  provides  sehedulers  with  more  flexibility  in  seheduling  their 
airspaee  and  redueing  their  workload. 
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Audit  Response.  While  we  aeknowledge  that  the  Air  Foree  has  worked  to 
improve  MAMS,  the  results  of  our  user  survey  support  our  eonelusion  that 
MAMS  still  does  not  have  the  funetionality  neeessary  for  them  to  do  their  job. 
Speeifieally,  out  of  159  survey  respondents,  105,  or  66  pereent,  stated  that 
MAMS  did  not  have  the  funetionality  needed  for  them  to  do  their  job.  Also, 

21  aeeount  holders  stated  that  MAMS  eauses  a  duplieation  of  efforts,  it  is  too 
labor  intensive,  Internet  dependent,  and  redundant. 

Allegation  4:  Many  of  the  users  do  not  have  access  to  the  proper  computer 
equipment  with  Internet  access  (unsubstantiated). 

Management  Comments.  The  Air  Foree  agreed. 

Allegation  5:  The  time,  money,  and  effort  used  in  building  and  training  for 
the  program  is  a  waste  (partially  substantiated). 

Management  Comments.  The  Deputy  Chief  of  Staff  disagreed,  stating  that  the 
MAMS  program  is  not  a  waste  beeause  it  was  not  intended  to  be  the  sole  system 
to  sehedule  SUA,  nor  was  it  intended  to  meet  every  single  user’s  unique  needs. 
Instead,  the  Air  Foree  built  MAMS  to  provide  an  eleetronie  eapability  to 
sehedule  SUA  where  none  existed.  In  addition,  the  other  DoD  seheduling 
systems’  interfaees  were  designed  to  give  the  Serviees  flexibility  in  meeting 
their  requirements. 

Audit  Response.  We  revised  our  audit  eonelusion  on  Allegation  5  in 
Appendix  B  to  elarify  that  the  money  the  Air  Foree  spent  on  MAMS  was  not  a 
eomplete  waste.  Instead,  we  state  that  the  money  the  Air  Foree  spent  on  the 
MAMS  had  not  been  effeetive  in  meeting  many  of  the  Serviee  seheduling 
requirements.  Additionally,  although  MAMS  may  be  able  to  traek,  eompile, 
and  produee  annual  usage  reports  to  the  FAA  and  it  may  be  the  only  system  that 
eleetronieally  eommunieates  with  SAMS,  after  10  years  of  development,  the 
Serviees  were  not  fully  using  MAMS.  Further,  the  other  DoD  seheduling 
systems  interfaees  with  MAMS  were  not  funetional  or  were  nonexistent. 

Allegation  6:  The  scheduling  process  through  the  system  is  cumbersome 
and  is  time-consuming  compared  to  procedures  already  established 
(substantiated). 

Management  Comments.  The  Deputy  Chief  of  Staff  disagreed,  stating  that 
users  must  enter  data  into  the  system  to  aeeurately  doeument  aetivity  and 
produee  reports  and  that  eomparisons  to  outdated  systems  are  not  valid.  He  also 
stated  that  MAMS  has  the  ability  to  aeeept  bateh  entry  via  spreadsheets,  whieh 
greatly  reduees  workload,  and  that  MAMS  performanee  was  well  within  the 
parameters  that  the  Air  Foree  established  in  the  operational  requirements 
doeument  for  the  rate  of  aeeepting  of  flight  requests.  He  further  stated  that  poor 
eonneetion  speeds  are  a  funetion  of  base  infrastrueture  and  are  not  a  refleetion 
of  the  MAMS. 

Audit  Response:  Our  audit  eonelusion  is  based  on  the  feedbaek  obtained  from 
100  of  193  aeeount  holders.  We  aeknowledge  that  poor  eonneetion  speeds  may 
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be  a  function  of  base  infrastructure  and  are  not  a  reflection  of  the  MAMS.  As 
discussed  in  the  Finding  section,  “Connectivity  and  Response  Time,”  the 
MAMS  Response-Time  Study  that  MITRE  performed  in  April  1999  stated  that 
the  cause  of  actual  or  perceived  bad  response  times  may  be  that  the  users’  work 
stations  are  incorrectly  configured.  Our  report  stated  that  Service  users  believed 
that  their  current  scheduling  systems  (not  outdated  systems)  worked  well  and 
were  easier  to  use  than  MAMS. 

Allegation  7:  The  utilization  reports  generated  from  the  MAMS  program 
are  not  sufficient  to  use  for  the  reports  we  send  to  other  agencies  requiring 
similar  but  more  detailed  information  (substantiated). 

Management  Comments.  The  Deputy  Chief  of  Staff  disagreed,  stating  that 
MAMS  was  designed  to  meet  the  reporting  requirement  of  the  Federal 
Acquisition  Regulation  (FAR)-73  and  FAA  Order  7400.2,  Ch  21,  Sect.  7. 
MAMS  also  provides  computer-based  support  for  efficient  management  of  DoD 
airspace  and  for  documentation  of  airspace  usage.  In  addition,  MAMS  has  the 
capability  for  units  to  tailor  their  reports  to  meet  their  requirements. 

Audit  Response.  Our  report  did  not  address  whether  MAMS  meets  the 
reporting  requirements  of  the  FAR  and  the  FAA  order.  Although  the  Air  Force 
response  stated  that  MAMS  has  the  capability  to  tailor  reports  to  meet  a  unit’s 
requirements,  the  report  states  that  the  MAMS  does  not  provide  the  tracking  or 
the  level  of  detail  needed  for  their  reporting  requirements. 
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Appendix  G.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Deputy  Under  Secretary  of  Defense  (Acquisition  Reform) 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications,  and  Intelligence) 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Army 
Chief,  National  Guard  Bureau 


Department  of  the  Navy 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 

Inspector  General,  Marine  Corps 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 
Chief,  Air  National  Guard  Bureau 


Non-Defense  Federal  Organizations 

Office  of  Management  and  Budget 
Department  of  Transportation 

Federal  Aviation  Administration,  Air  Traffic  Procedures 
Military  Operations  Specialist,  Special  Operations  Division 
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Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subeommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Serviees 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subeommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Serviees 
House  Committee  on  Government  Reform 

House  Subeommittee  on  Government  Effieieney,  Finaneial  Management,  and 
Intergovernmental  Relations,  Committee  on  Government  Reform 
House  Subeommittee  on  National  Seeurity,  Veterans  Affairs,  and  International 
Relations,  Committee  on  Government  Reform 
House  Subeommittee  on  Teehnology  and  Proeurement  Poliey,  Committee  on 
Government  Reform 
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Assistant  Secretary  of  Defense  (Command, 
Control,  Communications,  and  Intelligence) 
Comments 


OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 
6000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-6000 

June  1,  2002 


COMMAND.  CONTROL, 
eOMIAINICATIONS.  AND 
iNTGLUttNCE 

MEMORANDUM  FOR  INSPECTOR  GENERAL,  DEPARTMENT  OF  DEFENSE 
FROM:  DIRECTOR,  COMMUNICATION  PROGRAMS 

SUBJECT:  Audit  Report  on  Defense  Hotline  Allegations  Regarding  the  Military  Airspace  Management 
System  (Project  No.  D2001AD-0187.00) 


I  appreciate  the  opportunity  to  review  the  subject  draft  report  and  provide  the  following 
responses  to  the  IG  recommendations  to  the  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications,  and  Intelligence). 

Recommendation  1  .a.  “. .  .Task  the  Air  Force  to  establish  a  Military  Airspace  Management 
System  In-Process  Action  Team  with  representation  at  the  user  level  from  all  Services  to  conduct 
an  analysis  of  the  Military  Airspace  Management  System  to  determine  the  most  cost-effective 
scheduling  program  to  meet  DoD  future  airspace  schedule,  utilization,  and  report  requirements.” 

0ASD(C3I)  reply:  CONCUR. 

Recommendation  1  .a.b.  Suspend  any  future  installation  of  upgrades  or  technical  refreshment 
efforts  for  the  Military  Airspace  Management  System  until  completing  the  cost-effectiveness 
analysis. 

OASD(C3I)  reply:  CONCUR.  We  will  only  fund  the  Military  Airspace  Management 
System  operations  and  maintenance  costs  until  completion  of  the  cost-effectiveness  analysis. 

Recommendation  2.  “. .  .Task  the  Army  Deputy  Chief  of  Staff  for  Operations  and  Plans;  the 
Head,  Airspace  and  Air  Traffic  Control,  Departnlenf  of  the  Navy,  and  Director  of  Operations  and 
Training,  Deputy  Chief  of  Staff  for  Air  and  Space  Operations,  Headquarters  U.S.  Air  Force  to 
develop  and  implement  policies  and  procedures  to  standardize  the  Special  Use  Airspace 
scheduling  process.” 


OASD(C3D  reply:  CONCUR. 
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Department  of  the  Air  Force  Comments 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUAnTERS  UNITED  STATES  AIR  FORCE 
WASHINGTON  DC 


MEMORANDUM  FOR  ASSISTANT  INSPECTOR  GENERAL  FOR  AUDITING 
OFFICE  OF  THE  CvTSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 

SUBJECT:  Audit  Report  on  Defense  Hotline  Allegations  Regarding  the  Military  Airspace 
Management  System,  7  May  2002  (Project  No.  D2001AD-0187.00) 

1.  This  is  in  reply  to  your  memorandum  requesting  the  Assistant  Secretary  of  the  Air  Force 
(Financial  Management  and  Comptroller)  to  provide  Air  Force  comments  on  subject  report. 

2.  The  Draft  Audit  Report  on  Defense  Hotline  Allegations  Regarding  the  Military  Airspace 
Management  System  addressed  seven  allegations  and  made  two  recommendations.  Attached  are 
our  comments  on  the  allegations  and  recommendations. 

3.  If  you  have  any  questions  reference  these  comments  please  contact  my  POC  Capt  Will  Smith, 
HQ  AF/XOO-RA,  DSN  329  -0215,  commercial  (703)  601-0215. 


CHARLESF.WMftUeen,  USM= 

Attachment:  Deputy  (Mef  Of  8Mf 

AF  Response  to  Audit  Report  on  Defense  Holtline  Allegations  RegMfdlfiglfllj)|lllgllPI^  Airspace 
Management  System  (Project  No.  D2001  AD-0187.00) 


Audit  Report  on  Defense  Hotline  Aliegations  Regarding  the  Militai^  Airspace 
Management  System  (Project  No.  D2001AD-0187.00) 

Comments: 

ADegation  1:  Training  for  the  system  is  insuificient  (Partially  Substantiated). 

Concur  with  Comment.  Originally,  the  Joint  Program  Office  (JPO)  Hanscom  AFB 
provided  funding  for  training.  However,  as  resources  became  scarce,  the  JPO 
announced  that  funding  for  MAMS  training  would  be  eliminated  and  that  training 
would  have  to  be  funded  by  individual  Services  or  units.  The  Services  objected  to  this 
approach  as  none  had  budgeted  for  the  training  and  adequate  funding  was  not  readily 
available  to  accomplish  the  training.  With  the  decommissioning  of  MASMS,  an  Air 
Force  unique  system,  on  1  Oct  2000,  the  Services  agreed  to  give  a  few  of  the  US  AF 
bases  priority  for  training  scheduled  at  Raytheon  facilities  in  Lanham,  MD  prior  to  the 
MAMS  Central  Facility  (MCF)  standing  up  at  Tinker  AFB,  OK  in  Oct  2000.  This 
training  began  in  July  2000  in  order  to  give  USAF  adequate  time  to  complete  training 
and  become  familiar  with  the  system  once  they  returned  to  their  home  unit. 
Approximately  190  of  the  200  slots  available  were  filled  during  this  training 

MAMS  as  well  as  most  computer  tools  require  a  certain  level  of  familiarity  to 
maximize  the  training  opportunity  and  the  utility  of  the  product.  Several  actions  were 
required  of  the  students  prior  to  arrival  at  training  in  order  to  maximize  the 
effectiveness  of  the  training  provided.  The  trainees  needed;  a  valid  MAMS  account, 
and  knowledge  of  the  MAMS  user  Manual  and  their  unit’s  airspace  scheduling 
procedures.  Not  all  attendees  completed  these  actions,  which  sub-optimized  the 
training.  The  training  program  was  revamped  in  late  2000  and  none  of  the  above 
preparatory  actions  were  required  or  expected. 

The  MCF  established  at  Tinker  ATO  recognized  the  training  for  MAMS  was  not 
adequate  and  took  positive  steps  to  correct  the  deficiency.  One  of  the  first  steps  was  to 
ensure  students  were  prepared  for  training  when  they  arrived  at  Tinker  AFB,  by 
reinstalling  the  aforementioned  preparatory  items  left  out  by  the  training  program 
revision  in  2000.  Additional  steps  included  modifying  the  course  content  and  handouts. 
The  MCF’s  efforts  have  been  successftil,  according  to  student  critique  forms  collected 
by  the  MCF  following  training  classes  held  at  Tinker  AFB.  Since  Jan  2001,  responses 
indicate  86%  of  all  students  agreed  that  the  course  was  the  right  length  of  time.  Only  3 
(out  of  77)  students  felt  their  objectives  were  not  met  by  the  training  courses. 

Actions  to  be  taken:  Units  will  continue  to  fund  the  travel  and  per  diem  costs  of 
students  attending  the  training,  until  JPO  can  identify  funding  to  provide  training  at  no 
cost  to  the  user. 

Follow-up  report  due:  None. 
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Allegation  2;  Services  and  scheduling  agencies  are  not  using  the  system 
(Substantiated). 

Concur.  USAF  units  are  not  fully  utilizing  the  system  because  of  a  malfunction  of  the 
MAMS  conflict  detection  tool.  Following  identification  of  the  discrepancy  in  the 
conflict  detection  tool,  HQ  USAF  XOO,  Director  of  Operations  and  Training  issued  a 
directive  to  the  USAF  to  enter  data  into  MAMS  to  exercise  the  system  and  capture 
airspace  usage,  while  manually  scheduling  and  deconflicting  missions.  The  conflict 
detection  tool  discrepancy  has  been  repaired  and  has  proven  reliable  in  testing  and 
limited  field  use. 

All  new  airspace  scheduling  systems  under  development  will  interface  with  MAMS 
through  the  Other  DoD  Scheduling  Systems  (ODSS)  interface.  Existing  AF  airspace¬ 
scheduling  systems  are  in  the  process  of  being  modified  to  utilize  the  ODSS  interface. 
This  approach  is  consistent  with  the  MAMS  CONOPS  dated  1  September  1998. 

Aetions  to  be  taken:  Deputy  Chief  of  Staff  for  Air  Operations,  Headquarters  U.S.  Air 
Force  will  continue  to  provide  guidance  that  reflects  the  evolving  airspace  structure  and 
MAMS.  AF  continues  development  of  modifications  to  existing/new  systems  to 
interface  with  MAMS.  Once  interfaces  are  complete,  MAMS  will  extract  utilization 
data. 

Follow-up  report  due;  September  30, 2002. 

Allegation  3:  The  program  is  being  asked  to  do  too  many  functions  for  too  many 
different  types  of  airspace  (Substantiated).  ”Of  159  survey  respondents,  105  or  66 
percent,  stated  that  MAMS  did  not  have  the  functionality  for  them  to  do  their  job.” 

Do  not  Concur.  According  to  the  CONOPS  “MAMS  is  a  software  tool  that  provides 
the  capability  to  efficiently  schedule  airspace,  accurately  document  utilization,  interface 
with  the  FAA  Special  Use  Airspace  Management  System  (SAMS),  and  report  the 
utilization  of  airspace  to  the  FAA.  MAMS  performs  those  functions  as  well  as 
identifies  conflicts.  Once  individuals  realized  the  utility  of  the  system  and  information 
it  stored  they  started  submitting  suggestions  improving  its  ease  of  operation  and 
attempting  to  expand  the  system  to  fulfill  other  requirements.  Some  of  these 
suggestions  include  scheduling  subsurface  operations  and  tracking  expended  ordnance. 

Numerous  improvements  have  been  made  to  MAMS  in  response  to  user  requests.  The 
MAMS  Configuration  Control  Board  (CCB)  oversees  the  process  to  evaluate  user 
comments  and  impact  to  the  system.  The  CCB  approves  improvements  that  enhance  the 
capability  of  the  MAMS  to  meet  its  primary  goals. 
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The  MAMS  CCB  has  reviewed  a  total  of  131  user  comments  since  their  first  meeting  in 
Dec  2000.  To  date,  60  user  comments  have  been  implemented  and  closed,  28  are  still 
in  some  state  of  development,  and  9  remain  on  hold  for  future  upgrades.  In  addition,  the 
CCB  reviewed  a  large  number  of  problem  reports  to  prioritize  the  actions  to  be  taken. 
463  (or  83%)  of  558  problem  reports  logged  since  Sep  2000  have  been  corrected  and 
closed.  Corrective  action  on  55  problem  reports  is  currently  in  development,  and  10 
problem  reports  remain  on  hold. 

SUMMARY  OF  MAMS  PROBLEM  REPORTS  AND  USER  COMMENTS 

SINCE  SEP  2000 


CLOSED  REJECTED  ON  HOLD  IN  DEVELOPMENT  TOTM. 


USER 

60 

34 

9 

28 

131 

COMMENTS 

%  OF  TOTAL 

46 

26 

7 

21 

100 

PROBLEM 

463 

30 

10 

55 

558 

REPORTS 

%  OF  TOTAL 

83 

5 

2 

10 

100 

Additionally,  MAMS  offers  several  features  designed  to  provide  the  scheduler  more 
flexibility  in  scheduling  their  airspace  and  reduce  their  workload.  As  an  example, 
airspace  information  in  MAMS  is  the  same  as  it  is  in  the  Flight  Information 
Publications  (FLIPs)  and  Digital  Airspace  Flight  Information  File  (DAFIF).  If  airspace 
is  subdivided  in  the  FLIPs,  it  will  be  subdivided  in  MAMS  and  vice  versa.  The  MAMS 
user  (typically  the  Airspace  Manager)  may  create  dynamic  airspace  to  accurately  reflect 
actual  use.  An  example  is  Sells  Military  Operations  Area  (MOA)  (Sells  1/LOW).  Sells 
MOA  is  pubhshed  in  the  FUPs  as  a  single  airspace,  however,  it  is  locally  subdivided 
into  5  smaller  areas  to  support  the  training  requirements  of  the  airspace  users.  The 
subdivisions  are  created  dynamically  in  MAMS  (by  the  user)  and  a  relationship  (called 
a  dependency)  is  created  with  the  larger  Sells  MOA.  When  a  subdivision  is  scheduled, 
the  dependency  automatically  causes  the  entire  MOA  to  be  scheduled,  therefore, 
airspace  schedule  and  utilization  information  is  reported  to  the  FAA  in  the  required 
format. 

Dynamic  airspace  can  also  work  the  other  way.  When  several  smaller  areas  are 
routinely  sch^uled  together,  a  dynamic  airspace  may  be  created  encompassing  all  the 
airspace  and  the  dependency  may  be  built  so  MAMS  will  schedule  all  the  airspace  by 
using  the  single,  dynamic  airspace. 

Using  dependency  relationships,  MAMS  users  may  activate  and  deactivate  multiple 
areas  as  a  single  entity,  or  a  larger  body  of  airspace  as  multiple  sub-areas.  Conflict 
detection  and  resolution  follow  similar  rules,  such  that  even  if  a  user  scheduled  several 
dependent  dynamic  airspaces  and  there  was  a  conflict,  MAMS  would  only  generate  one 
alert. 
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Actions  to  be  taken:  None 
Follow-up  report  due;  None 

Allegation  4:  Many  of  the  users  do  not  have  access  to  the  proper  computer  equipment 
with  Internet  access  (Unsubstantiated). 

Concur. 

Actions  to  be  taken:  None 
Follow-up  report  due;  None 

Allegation  5:  The  time,  money  and  effort  utilized  in  building  and  training  for  the 
program  is  a  waste  (Partially  Substantiated). 

Do  not  Concur:  MAMS  is  a  top  down  driven  acquisition  in  response  to  a  GAO  report 
of  1987,  and  performs  four  main  tasks,  schedules  airspace,  interfaces  with  the  FAA 
SAMS,  documents  airspace  utilization,  and  has  the  capability  to  report  the  utilization  of 
airspace  to  the  FAA. 

MAMS  was  not  intended  to  be  the  sole  system  to  be  used  to  schedule  all  Special  Use 
Airspace,  nor  was  it  intended  to  meet  every  single  user’s  unique  needs.  MAMS  was 
built  to  give  units  an  electronic  capability  where  one  did  not  exist,  part  of  it  includes  a 
generalized  scheduling  tool  that  can  be  utilized  by  units.  Additionally,  the  ODSS 
interface  was  designed  to  give  the  Services  flexibility  in  meeting  the  Service/unit 
unique  requirements. 

Pursuant  to  the  GAO  report,  the  FAA  and  DoD  have  taken  different  approaches  to  meet 
their  varying  needs,  while  working  together  to  establish  an  interface  that  transfers  the 
required  information  between  the  organizations.  The  MAMS  system  interfaces  with  the 
FAA’s  SAMS  system  via  the  MAMS  -  SAMS  interface.  MAMS  is  the  only  system  in 
existence  that  electronically  communicates  with  SAMS. 

MAMS  can  accurately  track,  compile,  and  produee  annual  utilization  reports  that 
comply  with  the  format  standards  stipulated  by  FAAO  7400.2.  MAMS  accomplishes 
its  four  main  tasks  and  is  the  only  currently  available  system  that  performs  all  of  those 
functions  on  a  database  that  is  as  large  and  complex  as  the  United  States  Special  Use 
Airspace  and  Airspace  for  Special  Use,  therefore  it  is  a  not  a  “waste”. 

Actions  to  be  taken:  None 

Follow-up  report  due:  None 
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Allegation  6:  The  scheduling  process  through  the  system  is  cumbersome  and  is  time 
consuming  compared  to  procedures  already  established  (Substantiated). 

Do  not  Concur.  In  order  for  MAMS  to  accurately  document  activity  and  produce 
reports,  the  data  must  be  entered  into  the  system  at  some  point.  Comparisons  to 
outdated  systems,  or  systems  no  longer  in  use,  are  not  valid. 

MAMS  has  the  ability  to  accept  batch  entry  via  EXCEL  spreadsheets,  which  greatly 
reduces  workload.  This  upgrade  was  in  response  to  user  comments.  It  may  take  up  to  5- 
10  minutes,  or  more  for  less  experienced  users,  to  create  10  missions  in  the  main 
MAMS  application  (21  steps  per  mission).  It  takes  less  than  5  minutes  to  do  the  same 
via  the  spreadsheet.  For  units  that  already  use  a  spreadsheet-based  scheduling  product, 
the  process  is  even  simpler.  Luke  AFB,  for  example,  can  import  and  approve  over  160 
missions  (a  typical  daily  schedule)  in  15  simple  steps  that  require  only  about  two 
minutes  to  complete.  The  conflict  detection  process  then  runs  in  the  background  so  the 
scheduler  can  go  about  other  duties  (even  on  the  same  computer  system). 

The  Joint  AFFSA-DoD  Operational  Requirements  Document  for  a  Military  Airspace 
Management  System  AFFSA  001-85-I/n  ACAT,  DATED  20  Sep  93,  Capabihties 
Require  (4d),  (2),  Cost  and  Operational  Effectiveness  Analysis  (COEA),  states  that 
85%  of  the  requests  shall  be  accepted  within  2  minutes  (including  communications 
time).  Based  on  the  above  scenario,  the  per-mission  request  time  is  1  minute,  well 
within  ORD  parameters. 

Differences  in  network  load  at  various  locations  does  have  an  impact  on  MAMS 
responsiveness,  but  poor  connection  speeds  are  a  function  of  base  infrastructure  not  a 
reflection  of  the  MAMS  system.  There  have  been  numerous  memos  addressing  this 
issue.  Additionally,  the  MCF  can  assist  in  identifying  and  resolving  connectivity 
problems.  Since  1  Oct  2000,  the  MCF  Help  Desk  has  taken  only  12  calls  from  users 
reporting  MAMS  was  running  slowly.  All  of  those  reports  were  traced  to  problems 
with  the  network  at  the  user’s  end.  In  the  Luke  AFB  example  above,  running  conflict 
detection  on  160  missions  at  Luke  AFB  takes  nearly  40  minutes,  whereas  the  same 
schedule  can  be  run  at  Tinker  AFB  in  about  15  minutes.  Additionally,  the  Mitre 
Corporation  conducted  a  study  in  1999  that  addressed  connectivity  to  MAMS,  The 
study  reported: 

The  Military  Airspace  Management  System  (MAMS)  is  an  internet-based  application 
used  for  scheduling  and  reporting  the  use  of  the  Department  of  Defense  (DoD) 
Special  Use  Airspace.  This  report  details  the  results  of  a  study  undertaken  to  identify, 
with  the  MAMS  end-to-end  network,  the  source(s)  of  poor  response  experienced  by 
some  users.  The  conclusion  was  that  if  a  user  is  experiencing  significant  and 
persistent  delays,  the  source  of  the  problem  must  be  in  the  local  communications 
infrastructure  at  the  user’s  site.  Analysis  and  timing  measurements  indicated  that:  (1) 
the  MAMS  server,  (2)  the  communications  inflrastructure  (including  its  links  to  the 
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Internet)  at  the  MAMS  Server  site,  or  (3)  the  Internet  itself,  could  not  be  the  cause(s) 
of  persistently  poor  performance. 

Actions  to  be  taken:  None 

Follow-up  report  due:  None 


AUegation  7:  The  utilization  reports  generated  from  the  MAMS  program  are  not 
sufficient  to  use  for  the  reports  we  send  to  other  agencies  requiring  similar  but  more 
detailed  information  (Substantiated). 

Do  not  Concur.  MAMS  was  designed  to  meet  the  airspace  reporting  requirements  of 
Federal  Air  Regulation  (FAR)-73  and  FAA  Order  7400.2,  Ch.  21,  Sect  7.  The  ODSS 
interface  was  created  to  allow  Service  unique  requirements  to  be  met  by  other 
existing/future  systems  while  allowing  MAMS  to  extract  the  utilization  data.  Different 
airspace  may  have  additional  reporting,  accounting  and  tracking  requirements  based  on 
their  usage  and  chain  of  command.  MAMS  also  provides  a  custom  report  cap^ility 
wherein  an  airspace  manager  can  design  a  report  tailored  to  meet  that  unit’s 
requirements.  The  Concept  of  Operations  (CONOPS)  for  the  Military  Airspace 
Management  system  (MAMS)  dated  1  September  1998  states  that  “The  mission  of 
MAMS  is  to  provide  computer-based  support  for  efficient  management  of  DoD 
airspace  and  for  documentation  of  airspace  utilization”. 

Actions  to  be  taken:  None 

Follow-up  report  due:  None 
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DoD  IG  Recommendation: 


A.  We  recommend  that  the  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications,  and  Intelligence)  task  the  Air  Force,  as  the  executive  agent 
for  the  Militaiy  Airspace  Management  System,  to; 

1.  Establish  a  Military  Airspace  Management  System  In-Process  Action  Team 
that  includes  representation  at  the  user  level  from  all  Services  to  conduct  an 
analysis  of  the  Military  Airspace  Management  System  and  determine  whether 
it  is  the  most  cost-effective  scheduling  program  to  meet  DoD  future  airspace 
schedule,  utilization,  and  report  requirements.  The  In-Process  Action  Team 
should: 

a.  Identify  each  Service’s  future  requirements  for  Special  Use  Airspace 
scheduling,  utilization,  and  reporting; 

b.  Evaluate  any  new  technology  available  for  scheduling  and  reporting 
utilization  data  of  Special  Use  Airspace  to  the  Federal  Aviation 
Administration;  and 

c.  Consider  whether  it  would  be  more  cost  effective  if  each  of  the 
Services  use  their  current  scheduling  system  and  develop  an  interface 
directly  with  the  Federal  Aviation  Administration. 

2.  Suspend  any  future  installation  of  upgrades  or  technical  refreshment  efforts 
for  the  Military  Airspace  Management  System  until  completing  the  cost- 
effectiveness  analysis. 


Concur  With  Comment 

The  AF  concurs  with  the  establishment  of  a  Military  Airspace  Management  System  In- 
Process  Action  Team.  This  Team  should  become  a  function  of  the  established  PBFA 
Airspace  Sub-Group.  Representation  from  the  user  level  from  all  Services  should  be 
included  in  this  team  as  well  as  additional  expertise  as  needed  to  determine  available 
technology  and  assist  with  cost  effectiveness  analysis.  This  team  should  also  establish 
guidance  to  ensure  adequate  funding  is  procured  to  support  the  most  cost  effective 
Military  Airspace  Management  System. 

Even  if  proven  cost  effective  the  AF  would  be  hesitant  to  concur  with  other  Services 
interfacing  directly  with  the  FAA,  due  to  safety  concerns.  As  Services  continue  to 
manage  their  SUA  more  effectively,  and  the  demand  for  SUA  increases  more  Services 
will  utilize  other  Services  airspace  instead  of  acquiring  Service  specific  airspace  to 
fulfill  their  requirements.  MAMS  assists  the  Services  in  deconflicting  airspace 
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schedules  before  sending  them  to  the  FAA.  This  enables  the  FAA  to  refer  to  one  sole 
source  of  information,  instead  of  several  to  determine  the  status  of  SUA. 

The  MAMS  -  FAA  Special  Use  Airspace  Management  System  (SAMS)  interface  is  the 
only  electronic  gateway  in  existence  that  interfaces  the  military  airspace  scheduling 
systems  with  the  FAA  SAMS  system.  The  SUA  schedule  information  provided  to  the 
FAA  via  the  MAMS  -SAMS  interface  is  a  critical  element  in  the  FAA’s  National 
Airspace  System  modernization  effort.  The  FAA  provides  the  commercial  users  of  the 
system  this  SUA  schedule  information,  which  enables  aircraft  operators  to  flight  plan 
the  most  efficient  routes  around  active  SUA. 

The  AF  concurs  with  suspending  the  installation  of  any  major  upgrades  for  MAMS 
until  completing  the  cost-effectiveness  analysis.  However,  minor  upgrades  and 
technical  refreshment  efforts  are  requited  to  ensure  the  system  can  continue  to  interface 
with  the  SAMS  and  remains  operational.  MAMS  was  built  with  Commercial  Off  The 
Shelf  (COTS)  components  and  software  to  save  initial  costs.  As  the  components  and 
software  age,  the  vendors  stop  supporting  them  in  order  to  support  their  new  products. 
Therefore,  it  is  imperative  that  MAMS  is  upgraded  at  the  same  pace  of  technological 
innovation  to  ensure  it  remains  supportable  and  does  not  fail  for  any  period  of  time. 
Previous  leadership  recognized  the  importance  of  upgrading  software  and  specifically 
added  a  portion  to  the  MAMS  Concept  of  Operations,  signed  1  Sep  98,  stating  the  MCF 
will  maintain  COTS  equipment  and  software  and  will  perform  software  upgrades  in 
response  to  user  requirements. 

The  estimated  completion  date  of  the  MAMS  In-Process  Action  Team  evaluation  is 
July  2003. 


DoD  IG  Recommendation: 

B.  We  recommend  the  Assistant  Secretary  of  Defense  (Command,  Controi,  and 
Communications,  and  Intelligence)  task  the  Army  Deputy  Chief  of  Staff  for 
Operations  and  Plans;  the  Head,  Airspace  and  Air  TrafGc  Control,  Department  of 
the  Navy;  and  Director  of  Operations  and  Training,  Deputy  Chief  of  Staff  for  Air 
and  Space  Operations,  Headquarters  U.S.  Air  Force  to  develop  and  implement 
policies  and  procedures  to  standardize  the  Special  Use  Airspace  scheduling 
process. 

Concur. 

The  Air  Force  already  complies  with  this  lecommendation.  According  to  the  DoD 
Draft  Audit,  Section,  “Factors  Effecting  Use  and  Acceptance  of  MAMS”  subsection, 
“Service  Pohcies  for  Standardizing  Scheduling  Process”  “With  the  exception  of  the 
Air  Force,  the  other  Services  lacked  policies  and  procedures  for  standardizing  their 
SUA  schedule  process,” 
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Deputy  Chief  of  Staff  for  Air  Operations,  Headquarters  U.S.  Air  Force  will  continue  to 
provide  guidance  that  reflects  the  evolving  airspace  structure,  and  technological 
advances. 
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